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•This is, the summary of Phase I of the Evaluation Practices in IG^ schools. 
Phase I was a large sample survey of 156 schools. Two Instruction and Re- 
search (I & R) units per school • were studied — one that included grade 2 
children and one that included^ grade .5 children. The fp^lqwing were all 
hypothesized to be positively correlated with measures of student achieve-' 
merit in reading and mathematics: ^ measures of the organization of classroom 
instruction, a measure of the use of varied curriculum materials designed to 
be compatible with the- InstijUctional Programming Model (IPM) , a measure of 

the extent of sclioolwide implementation of the IPM, and measures of assorted 

/ 

schoolwide organizational* features distinctive to IGE. There were al^o 
hypothesized links among some of these measures. Although some of the 
hypothesized links ajnong IGE-related measures* -were empirically sup^rted, none 
of- the hypothesized -links witih^student achievement were supported by the data. 
-Some of t^ese "negative findings' can plausibly be attributed to measurement 

m 

error and specification error inherent to the survey methods used in Phase I; 
others cannot, Soi!he organizational features distinctive to IGE were found to 
be positively correlated with teachers' job satisfaction. 



The IGE 'Evaluatio n Project ^ • , . 

" — : ■ ' V 



This is a summary repo.rl^ on schooling .practices of a sample of schools 



which wdfe identified as implementdrs of Individually Guided' Education (IGE). 
;t is the summary of Ph^se I^ \7hich is one of five relAted phases of. an ex- 
tensive 3tudy designed to evaluate IGE. 'in the following pages, the purpose 
j^and de^i^n o*f the IGE Evaluation Project are described. 

Through the ,qombiried Efforts ^of the Wisconsin Research and Development 
Center for Individualized Schooling- (R&D Cente^r) , the Oniversity of Wisconsin 
IGE Teacher Education PrcJject, the Kettering Foundation (/r/D/E/A/) , and IGE , 
co<^dinators ' in 25 states'^ mor^ than 2,000 eilementary schools adopted* a system 
of elementary educcition called Individuality Guided Education (IGE), However* 
prior to the IGE Evaluation Project, no comprehensive picture, existed which 



showed how extensivel^y or how effectively IGE had been impl^entied in thesfe 



schools. 



" Thus, the purpose of the IGE Evaluation Project was to evaluate *IGE in 

.order t4 gain a more comprehensive view of the system's operation and effec- 
. ** . • * ^ ♦ 

tiveness. The desired outcome was to identify which features of flndividualized 

• 4 

schooling contribute most to the success of reading and mathemafics instruction, 

♦ 

It was assumed that the features we wduld identify could -be related^o the R&D 
Center's emerging theory of schooling. In addition, methodological, procedures 
and paradigms used in the study would be of general ifttere^t to other scholars 

V ^' ' ' ■ ^ ' 

studying ^schoo3,ing practices. 

For more than a decade, IGE was the primary program of the R&D Center. 
It is a complex system based on theoretic and pragmatic ideas about schooling. 
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Children*' s learning, land' the professional roles 'of sch6ol staffs. The IGB 
program* -^ntains' seven components: . . 

li». Multxunit elementary school organization/ * • * 

. 1 . * 

2. Instructional programming for the individual student, 

lb 

3. > Assessment^ an3 evaluation %f or educational de*cision making, 

■ ' - , • *' » ' . " ^ . ^ 

4.. Curriculvyn and instructional materials and apti,vities for each child ts 

instructional .program, - , • ' . ""-^'^ 

f ' 5» tIome-school-*cpmmunity -relations programs,* « '* * ' » * 

* ♦ ' • . 

t * . , . .* . 

6. Facilitative envircmmehts for professional -growth, and 

7.. Continuing r'esearch an(5 development for system improvement, 

Eaph of these seven components is thgkresult 6t a long, collciborat;ive study 

■ \ 

by various scholars and D;:of essional educatorJb. ; ' t . 

^ ' Four types of variables were identified to guide the evaluation of IGE: 
pupil and staff outcomes, means of instruction, support systems, and piipil and 
staff backgraund. Vfith these type^ of variables in mind, a descriptive frame- 
work, was developed that Considers outcomes of: I(3e as ^ function of both the 
means of instruction and the degree of implementation (Romberg, 1976). Figure 1 
'shpws a ^e'neral a^ 'priori. framewdrR of how th.e four types of variables are 
related.* 

i- Pupil and staff outcomes, and the extent to which thesd outcomes have 

• — 

been attained, should be the initial basis of an IGE evaluation ^ As Klausmeier 
(1977) stated: 

Students, upon completing IGE elementary schooling, should have 
achieved more than in other kinds of schools, should have acquired 
higher-level conceptualizing skills and other abilities which 
enable them t6 continue* to learn, and also should have developed 
healthy self -concepts. (p* .7). . * . * 
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Presumably, chaoges in , pupil outcomes are', in part, a result* of changed , 
teachefr activities. For that reason^ any plan for the ^valuation of IGE « 
should give attention 'to staff outcomes — includis^g increases in staff knowl- 
edge and changes in attitudes and beliefs. .* ^ ♦ ' 

Both pupil and staff .outcomes are illustrated in Figure 1 as being multi- 

variate and multilevel. Pupil outcomes include achievement in reading and 

mathematics, cognitive skills such as conceptualizing and problem solving, and 

self-direction. Staff outcomes include attitudes about children a^d schooling, . 

beliefs about the nature and value pf education, and knowledge of IGE principles 

pertaining to individual differences among children and use of the Instructional 

f 

Prograirming Model. • . 

2'. The instructional means of formal schooling must be a second basis 
of an IGE ev^uation . It has been fashionable in evaluation circles' to con- 
centrate on ends or outcome^ and to ignore the means by which they are reached. 
Reform movements, such as IGE, invariably involve criticisms of and changes in 
means. Consequently, judging the value of the means is as important as assess- 
ing outcomes.' ' ^ 

Means of instruction were separated into three sets of activities based 
upon the operating characteristics of IGE schools: (1) staff activities of 
the Instructional Improvement Committee (IIC) and the Instruction and Research 
Units (I & R Units) , (2) activities of the staff teacher (in both curriculum 
pianagement and pupil interactions) , and (3) activities of pupils as they are] 
related to reading*-an4 mathematics instruction. 

3. The degree to which the support systemg of o:ge have been incorporated 
and developed in a school must be judged . The seven components of IGE have 



/ 
1 



evolved features of schools intended to support n*ew instructional methods 
and thereby to produce desired pupil and staff outcomes. It can be argued / 
that the effectiveness of an IGE school depends upon which components Qf IGE 
have been .implemented and how well ,tho;se components are operating. 

The support systems for an IGE learning environment were separated into* 
four categories' in Figure 1. The first includes the multiunit organization, 
instructional programming, and evaluation (IGE Components 1, 2, and 3).* The 
second category, curricular materials compatible with instructional programming 
and evaluation (IGE Cpmponent 4) , is represented (but not exhaustively covered) 
by the three major curricular products developed for IGE. These are the 
'Wisconsin Design for Reading Skill Development (WDRSD) ' (Otto, 1977), Developing 
Mathematical Processes (DMP) (Roiri)erg, 1977) , and the PreReading Skills Program 
(PRS) (Venezky & Pittelman, 1977). The third category of support systems in- 
eludes home -school -community relations, facilitatiye environments, and continued 
research and development (Components 5, 6, and 7). The final category of sup- 
port systems includes the teacher training and IGE staff development programs. 

4* The fourtlj^^-type of variable iir-Pigure 1 is pupil and staff background . 
Background)mriables, were included becaus^ knowledge of pri©r pupil achievement. 



level yc^f motivation, and learning style is assumed to be necessary Wr efficient 

grouping of students a^nd selection of appropriate activities. Similarly, staff 

^^^^ 

experience with IGE principles, with working in groups, and with the pupils of 
the particular school is important. 

The functional rl'lationships illustrated in Figure 1 convey the following 
premises: (a) the degree to which IGE support systems havd been implemented, 
together with pupil and staff backgrounds, directly influences the means of » 



instruction in^an IGE school; and' (b) the means of instruction, along with pupil 
and staff backgrounds, fccount for pupil and staff ou^cpmes. 

To clarify how .this Pha^ I report fits into the overall study, all ' 
five phases of the project are described below. . ' 



Phase I 



Phase I consists of a large sample study which provides basic information^ 
'about IGE schooling. Certain* features of IGE schooling have beep reputed to* 
be crucial to IGE success • The purpose^ of Phase I was to examine the extent 

^ / • • • 

to which those presumably essential 'features^^ave been implemented among IGE 
schools and to assess the effectiveness of that implementation. In this large 

s^ple study (including approximately 155 IGE schools), information was 

'? . ^ " 

gathered using self-report surveys from IGE school staff metobers and standard, 

paper-and-pencil instriiments from students. The data were intended to pro^jide 
a functional understanding of IGE features, processes, and outcomes by relating 
a broad scope^^of ^ar iables in an interpretive manner. Using structural equa- 
tions. Phase I simultaneously examines relationships among the network of 
variables believed, to influence meaps ^of instruction, staff outcomes) ^ and 

pupil ' outcomes — thereby advancing our understanding of each IGE feature and 

• * 

the network as a whole. , - " 

Phase II ' • ^ 



"pfiase II verifies the self-rfeport: data, gathered in Phase I and extends 
data collection to include a fuller range of variables. As verif icatiorf^'of 
the Center's Phase . I work. Phase II was conducted by •Research Triangle Insti- 



tute, Roderick A. Ironside, principal investigator, under a' sulpcontract from 
the R&D Center. The final report is available as Technical Report No. 499 
(Ironside & Conaway, 1979). Specifically, the objectives of Phase II were: 

1-. to determine^ the validity of the self-report data gathered in Phase I 

2. to use interview arid observat^ion data to extend the information^ about 
each category of. variable. ' , • , <» . 

3. to ascertain the role of developmental agencies in the national dif- 
fusion process as perceived in IGE schools. 

4". to gather cost, data so that some indications of , co$t-ef fectiveness 
could be determined. _ 

The areas <^f cost an^ of implementation history, including the role of 

developmental agencies, are the primary additions of Phase II to Phase I data. 

The importance of cost* amalysis has been discussed by Rossmiller and Geske 

'(1977) S Adoption and institutionalization of innovative practices are pro- 

cesses 1 which interest not only practitioners and scholars, but also funding 
• ^ «<» - 

agencies (Becman & McLaughlin, 1976) . ^ 

Phase III , * . 

Phase III focuses on the social meanings which emerge as IGE is used on 
a ^ay-to-day basis. This phase seeks to increase understanding of thB impact 
of educational reform by viewing schools as having a "culture" which governs*" 
their members.,. According to^ this approach, a school is a complex socicil - 
"arrangement With underlying patterns of conduct. that channel the thought and 
action of persons in the school. These underlying cultural'* patterns should 
not be ignored. Failure of educational,..planners to consider the cultural 
patterns of schools has produced unanticipated and unintended results from 



reform 'efforts. Research into the impact of educational reform suggests 
that changes in school programs f^quently do not alter existing educational 
patterns-. Instead, the reforms are transfomifed tb: fit into the everyday patterns 
of school life. Early in the development of IGE, the R&D Center explicitly 
stated that the purpose of IGE schooling was to alter the^. substantive hatiJre 
of curriculum and instruction in Elementary schools (Klausmeier, Morrow, &- 
Walter, 1968). It was , argued that, as schools incorporated IGE, oew patterns of 
conduct, values, and attitudes would emerge. In part, IGE was expected to 
instill a sense of intellectual vitality, critical^^ thought, and personal re- 
sponsibility into student^' work. From a different perspective, IGE was ex- 
. pected to infuse the staff with a sense* of professional participation — an 
active searching for new information and new skills. But, ciaes the implementa- 
tion of IGE in fact;'alter teachers' conceptions of their jobs and students' 
conceptions of learning? Do new values, * attituijes^ and patterns of conduct 
develop in IGE school^, or are>old underlying patt^nis disguised in new forms? 
Phase III sought answers to these questions* and thereby helped to prpvide a 
comprehensive perspective for understanding the impact c5f IGE'^pon schooling. 

Phase ly , ^ • . ' / \ . 

V 

Phase IV was designed to evaluate the effectiveness of WDRSD, DMP, and 
PRS a^ aids to instructional programming and to examine the relationships 
of means of instructioi^ and instruQj:i:6nal time to pupil outcomes. The success . 
of IQE depends heavily on the availability of curricular materials compatible 
^ with instructidnal programming -for the . individual student ahd the availability 
of appropriate evaluative procedures (Klausmeier, Rossmiller, & Saily, 1977) ^ 
The three programs (developed at the R&D Center — the Wisconsin Design for 

■ ^ 18' ■ 



Reading Skill Development (WDRSD) , Developing Mathematical Processes. (DMP) , 
- and PreReading Skills Program (P^)-rwere designed to meet this need. 

Phase V . • ' ■ • 

Phase^ V synthesizes the results of Phases I through IV of the IGE Evalua 

^ ^•'^'^^ ' 

tion. Each phase of the evaluatrlorf.^^tudy was designed to complement and 

strengthen the validity of the data gathered by the previous phases. For 
example, data/^on means of instruction, gathered by<the large-sample study of 
^Phase I, are examined in somewfiat greater depth in fewer * schools by the Phase 
II studies. Phase Ill's analysis develops a view of* instruction from a dif- 
ferent perspective. Phase IV explores 'means of instruction within specific 
curricular areas. 

Irfstead of merely adding togethe1\ summaries of the different evaluation 
phases. Phase V proposes to integrate and interpret thg data 'from. all the 
. phases ifito a series of statements about the implications of the project for 
contemporary educational issues. This phase explores seve^kl major' themes 
aboiit schooling by using data from all phases of the evaluation. * 



1 
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' ' I" ■ 

. ... . Introduction ■ ^ ~^ 

This document provides, a discussion of the relationships in Individually 
Guided Education (IGE) schools between the orgatiizational. and instructional , 
features of schools anckthe reading and matli •achieT^ement of second and f^fth 
gracje students. The document also discusses the relationship between organiza- 
tional features of IGE a^d teacher Job-satisfaction, The docutnent presents 
- 'data collected by the large sample survey component of the IGE evaluation, ' .\ • 

referred to hereafter as Phase I, The theoretical pjSitions that are p€rt of 

. ' / : * \ 

the .IGE model — soltie explicit and some i|nplicit "but evident— imply what some of 

the relationships should be. *rhus, we were able tq^ deverop a theoretical model ' ^ 

of how vajri'ations in different organizational and in^trjactional features would ^ 

be linked/ and how, in turn,.*hose variattons would be linked to variations in 

achievement. This report presents" thati|tneoretici3^< model and evaluates' how 

It; . • / 

/adequately it fits data .obtained from schools. 

The purpose of phase I, as with^ .the other IGE' evaluation phases,, is to 

' ' ■ > ' \ ^ ' ' • *' \ 

, q^ih a more cdjnprehensive vievf of the model's operatioiTand effectiveness. 

The basic objective is, to identify^f^atures'of IGE schooling that contribute . ^ 

to successful instrucjtion, especially in trading and mathematics. The iden- * ^ 

tific^tion'of such featur^, is oritic^O. to evaluating IGE as aa educational 

system; it is also integral to understanding' schcioling in general. • • 

^. * Certain features of IGE schooling have a reputation as keys to the^ progr^'s 

\ success. Phase I examioes v-ari^tions in the extent to which these- presumably ^ 

-essential -.features ha^ be'en implemented among IGE .schools, and assesses ^how. * - 



infiueiltiSi.'-^uth' variations in Implementation are/ 'J^s.M'^ large sample study 
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(including approximately 155 .aahc5bl,sj > Rhase I provides basic information 
about, IGE features, processes, ^^^^^tc-o^es ^py coating a broad scope of 
vj^rriables in an interpretive mann^^y^^e/ vafe5able^,are measures of organisa- 
tional features, instijuctional featureV/J^feij^ent achievement,, staff job satis- 
faction, and other characteristics of studams ^nd staff. The specific pur- 
poses of Phase I are: ^ ^-^ , ' * , 

1* to determine the degree* to-* which the seven IGE components have bee*n 

* ^ * \ , , . 

implemented in JGE schools; * . ^ 

^ 2. to describe and examine the 'relationship between the implementation 

of IGE components and means of instruction , particularly, in reading and math^ 
matics; . - ' * ' ' - ^ 

3. to describe find examine the relationship between the implementa^tion 
of IGE components and staff outcomes ; and ^ . ^ • 

4. to describe and examine the relationship--presumably by way of the' ^ 

means of instruction — between the implementation of IGE components and pupil ^ 

^- . * 

outcomes . This includes such outcomes^as reading and mathematics achievement, 
selected cognitive skills, and aspects of personality development. 

The means of instruction and ou|:come variables of this study are, without 
question, influenced by multiply, simultaneous causes that resist easy de- 
scription- Since causal relationsh^^ are easier to study in isol^Jifon, most > 
research has examined only one or a few of , these relationships. Simple com- 
pa^isons between IGE as an undifferentiated 'package"*, and other educational 
alternatives provide us with little information about specific * features and pAo- 
cesses that occur in. IGE schools- Therefore^ Phase I simultaneously e2S^lijWs 
relationships ^ong the network of variables believed to influence- means of --^ 



instruction, ^taff outconves, and pupil outcomes — thereby advancing understand- 

inq^of each IGE feature and the network of variables as a .whole. 

;*^ 

The Phase I study was not designed as a direc^ comparison between IGE' 

-^hools and other schools. Iristead, it is an assessment of certain fundamental 

f ^ • ' ' / 

ahd empirically testable premises on which IGE is based. IGE evolved as an^^ 

was disseminated as an educational sys^t^m with a distinctive approach to in- 

struct ion (succinctly Wscrib^d in the Instructional Programming Model) , dis- 

tinctiye ways of coordinating the instructional effotts ot teachers, distinc- 

^tive forms for involving teachers ip decision^ that affect the school as a 

jiS^hole, ai^d a supporting network of other IGE S9hools, regional coordinators, 

and^isseminators. Some curriculum paclcages were designed to lend themselves 

to use of the Instructional Programming Model. - IGE as its /ievelopers intended 

it was not ^ai) isolated innovation 'but a complete system. As a complete system, 

IGE is built upon theoreticaL positions about the goals of education', the 

effects Qf certain forms of instruction, the effects ot school organization 

on instruction, and the effects of linkage's that go beyond the walls of the^ 

school building. 

Some of the premises on which IGE is based are explicit. Others are im- 
^licit, but evident. Three general kinds of premise^ ' were recognized in the 
Phase I study. Those are: ^ . ' , 

' i. The in^^r^ctional practices associated with the Instructional Program- 
ming Nfodel mklce^igh student achievement more likely (Klausme'ier, Karges, &- 
Krupa, 1977, pp. 333-334). ' 

2. Certain systems of record keeping 'and information collection make i't 
more likely .that the instructional practices' of the Instructional Programming 
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r ' • ' 

rtodel will occur* Likewise^ the use of curriculum materials thAt lend them- 
selves to record keeping, information collection, «and segmentation of curricu- 
lum units make those instructional practices more likely to occur* ^ 

* 3. Certain features of school organization make use of the Instructional 

* • ♦ 

Programming Model more likely. Those organizational features also ma^, it 

more likely , that the, staff will be satis £ie<3 with their jobs— p^he 'influence 
of organizational . features on job satisfaction is considered at the en<f of 
this* report. 

It should be clear frpm this discu^^sion that Jthe IGE System of elementary 
education w^s based on a set of premises about supportive systems, insliruc- 
• tional means, and the impact of those upon both pupil and staff outcomes. 

In designing the revaluation plan, a descriptive frame work * re latp.ng the various 
aspects of IGE 'wiwLproposed (Figure 1). From this framework, a more specific 
model was developed *ancK variables were identified for aspects of the model, 
as described in the following sections. 

A . . ^ 

Pr'emises of the First Kind 

* ' 

The first kind of premise includes two instructional practices and their 
' reputed influence on student achievement. The first practice was character- 
i^zed in^the Phase I analyses by a variable named Management of- Grouping and 
Instructional Continuity, the second by a variable named Individualization 
of Instructional Decisions. These variables and others were scaled from in- 
formation drawn from several questionnaires treated as one large pool of 
potentially relevant items. 

The IGE ijiodel suggests that the organization of classroom instruction 
sh^ould have an effect on student achievement. The Instructional Programming 
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Model (IPM) is used to group Students according to their individual needs, 
mcUcing teacher-student instructional interactions more effective. The variable 
Management of Grouping and Instructional Continuity (IE) measures several 
'facets of the classroom instructional environment and should, according to 
the IGE modei, be directly related t^o student learning. 

The variable Individualization of Instructional Decisions ^(IDM) measures 
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the extent "to which instructional decisions take into account the individual 
needs of the student. Taking individual student needs into account is the 
cornerstone of the IGE model, and it is supposed to be related to studeht 
achievement. ^The individualization of instructional dedisions is also integral 
to the forms of instructional grouping mentioned before.' ^ 

The ^stulated causal links, between thesQ variables and measures of student 
achievement are shown in Figure 2. 'The figure expresses an IGE theory i^f the _ 
causal relationships among the variables. It is mednt to express an IGE theory, 
but it does not always use IGE nomenclature. 

Student achievement is the dependent variable in Figu3?e«^. Only Jfihi^e- 

ment in reading, and mathematics as measured by t^^ California Test of ^asd 

Skills (CTBS) are considered in this report. Other areas of achievement (e.g., 
/' ♦ 

science, social studies) , could also have been assessed, but because reading 

^nd mathematics were the two curricular areas in v^hich materials had been pre- 

* • *i 
pared for use in IGE schools, emd because most schools in the sample indicated 

they were implementing the IGE Instructional Prograinning^ Model in those - 

curricular areas, the/ decision to limit achieVemeht assessment to those areas - 

seemedvwajranted ; 
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Premises c^f the Second Kind ' • 

The second kind of premise involves procedures and features intended to 
facilitate desirable instructional practices. 

Three variables reputed to affect Management of Grouping and Instructional 
Continuity^ are shown in Figure 2« One of them has already been mentioned — 
Individualization of Instructidnal Decisions. The second is the use of a 



^^atiety of ^rriculum resources (UCR) , and the third* is the sc^ioolwide imple- 

mentation of the Instructional Programming Model (SIPM) . ^ Schoplwide implemen- 

> 

tation is, important because^ presumably, the instructional practices of, the 

. * u 

individual I & R units resonate with the practices elsewhere in the school. 

Factors influencing \he individualization of instructional decisions . The 

IGE model suggests that the amount and variety of student information available 

' ' ' ( 

shoxiid directly influencje the individualization of instructional decisions. 
The amount and variety of student information available is reflected "by the 
variable Information^.Acquisition (lA) . The model also suggests that school- 
wide implementation, of the IPM should promote such individualization. The 
procedures heeded for efficient and useful information acquisition are pre- 
sumably limited by incomplete school wide implenjen tation of the IPM.^^""^ 

premises of the Thjrd Kind 

The third kind of premise involves features of school organization that 
are distinctive to IGIU These .are organizational' features intended to make 
use of the Instructional Programming Model more likely. * ^ 

Factors influencing general implementation of the IPM . Two organizational 
features in' particular have beei> presented in the IGE literature a& ones which 
create an organizational -environment con4ucive to the IPM. The first of these 
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is an assortment of activities qollectively labeled and measured as Procedures 
Fostering Coordination and Improvement of the School Program (GOS) . The 
second, reflected by the variable Intra-Organizational Structure (lOS) , is a 
collection of structural arrangements distinctive to IGE schools, such as o2?r 
ganization into I & R. units, existence of an Instructional Improvement Committee 
and so forth. .0 , , • 

The effect of organizational features on schoplwide implementation of tl^e 
IPM would be difficult %o assess if teachers' backgrounds and beliefs were not* 
considered, too. Two appropriate staff" measures are included. The first. 
General Staff Background (06B) , is an aggregate measure of how much IGE-related 
experience teachers have. The second. Belief in Individual Differences (INDIV), 
is a measure -of how strongly^and unanimously the teachers of a school endorse a 
basic assumption of IGE>i— the assumption that students differ in ways that in- 
struction ought to talce ^nto account. > 

Factolrs influencing thg use of curriculum resources . One distinctive or- 
ganizational feature of IGE is the system of linkages between IGE schools. "One 
intended consequence of. such Inter-Organizational Relations (lOR) is the exdhang 
of information about IPM-compatible curriculum materials and other curriculum 
resources. Contact with other schools would presumaUly help teachers to us^e 
a 'ivariety of curriculum materials. The variety of cxirriculugi materials used 
by an J & R unit would Sepend partly on other practices. of the unit, too. For 
instance, I & R Cinits that gather and organize information about students, 

through -.their efforts to individualize instructional decisiona/ would 'be more 
^. ' , *^ . * 

likely to a variety of curriculum materials. 
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Siammary of the Model 

Each arrow shown in t^he diagram signifies a causal link assumed in the IGE 
system. Not every causal link assumed in IGE is indicated, however. Suffice 
it to ^say that the evaluation staff worked hard to create a model that ex- 
presses some of the major causal assertions implicit in IGE. The intent of the 
evaluation effort was not to ask whether IGE works. Rather, the intent Was to 
ask whether IGE works in the way its developers thought it would. Critics of 
'jbhis approach to evaluation might charge that it accepts, the definitions of 
the program- being 'evaluated. On the other hand, the approach can potentially 
demonstrate that the factual claims on which a program is based are not borne 
out empirically. The most obviously serious criticisms are thofee that meet a 
' model on its own empirical grounds. 

The Phase I study evaluates premises of the kind listed above by empirically 
assessing the relationships they imply. This approach to evaluation would not 
be feasible if IGE schools had uniformly implemented the organizational features, 
curriculum features, and instructional practices suggested by, experts in IGE. 
Such uniformity, however, does not exist, as is apparent from the distributions 
of variables. Phase III of the evaluation^ a field study of "a few highly re- 
puted ^IGE schools, provides corrobprating evidence that IGE SLchools dre not 
xiniform. In Phase III, marked variability was found even among schools reputed 
to be exemplary IGE schools. 

If'or the purposes of this study, th^e fundamental premises listed above have 
been represented as a network of postulated causal links among the variables 

of the study. Figure 2 presents these causal links in diagram form. It shows ^ 

\ '~' ^ * 

the, paths of influence assumed to underlie the relationships between reading 

achievement and the other variables studied. The^ figure expresses an IGE 
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theory of how each variable is causally related to the other variables. The 
formulation and logic of this model are discussed in the method section. 

The diagram is known as a structural model, structural diagram, path dia- 
gram, or causal model and follows certain graphic conventions. According to 
these conventions, a straight, unidirectional arrow signifies that the variable 
at^its base directly influences the variable at its tip. The omission, of an ^ 
arrow constitutes an explicit theoretical statement that no di^rect causal xe- 
lationship exists. A variable is an indirect cause of a dependent variable if 
a path through two or more arrows can be traced from the dependent variable 
back to the first variable. Associated with each straight arrow is a nonzero 
value. The sign of the value denotes whether an increase in the causal variable 
produces an increase (plus) or a decrease (minus) in the dependent variable. 
A curved, bidirectional arrow is^used at the left of the figure between vari- 
ables which are known to be coi^related, but for reasons not covered in the 
scope of the modql. 

• A ^system of structural* equations corresponds to the model . These equations 

f 

may be st^tis'tically examined for, their agreement with the data collected in 
the Phase I study. Within the limits imposed by measurement error in the 
group-administered, standardized ^ests and questionnaires- used to collect the 
data, this approach tests the theoretical model that underlies IGE. If the 
relationships between variables are not consistent with Figure 2, theti probable 
inaccuracies in the underlying* IC^model wili be revealed. On the positive 
side, this approach can indicate important features and pjsoe^jj^es which deserve , 
more attention front schools implementing IGE. In this context, "importance" 
mecins that a feature or process i.niluences outcomes that are ^socially valued. 



• . . II 

Methods 

* * 

Sampling ' , * • . * 

The popuration . The population with which this evaluation j.s concerned 
is constrained in several ways. The population is necessarily limited to those 
schools that define themselves as IGE schools. There were between 2^^pctl and 
3,000 such schools in 1974-75 (Klausmeier, 1977). The population studied by 
this evaluation is further constrained to include only those schools that re- 
sponded fully to the IGE schools questionrl^ire of March 1976. .^There were 946 
such schools, 768 of which had both second- and fifth-grade students. The 
evaluation was limited to students in second and fifth grades, and their ;. 

teachers. Second grade is the first primary grade in which group-administered 
[ ' , ^ * - * ' 

paper and pencil tests are sufficiently reliable. Fifth grade is the last 

elementary school grade commoh 'to all elementary schools-. This population of 

768 IGE schools is nartionwide; it includes urb^n areas, rural areas,- low- income 

areas, and high-income areas. ^ 

r 

Sample size . The structural equations analysis is a complex orie. Rela- 

' ' ' ^ " , ^ , 

tioilships between many variablestare explicitly^ and implicitly assess^ed* An 

cuialysis like this requires many degrees of freedom, hence many schools. In 

light of the plcinned analysis, the Phase 1 IGE evaluation staff decided to 
:^ * - . 

geek a salhple of^ 300 schools* ' » ^ ^ ^ * . 

• * • .* 

^- VSampling plao . To select the sample of 300 schools, a plan of stratified 

* • ^ . • ' 

random Sampling, was chosen, in preference to simple random sampling . In -hetero-' 
geneous. populations, where the. strata reflect relevant dimensions of jheterogeneity, 
stratified random sampling better ensures that small samples will represent the" 
full variabiliiy^of'^e population • The .use of stratified random sampling, 

^ 21 31 * 



however, presu^'poses that all members of the population (in this case 768 
schools) have'^been classified into mutually .exclusive strata > These strata, 
as imp^ed cibove, should each have memberships that are relatively homogeneous 
with respect to .the characteristic^ whose representativeness* is being ensured. 



stratification variables. Information ^about each of the 768 schools in 



the population was -obtained through the March 197*6 IGE Schools Questionnaire • 
That questionnaire concerned the implementation of various IGE features 
thought to be influential with respect to pupil outcomes, teacher outcomes, 
and aspects of the instructional process. That information was used to con- 
Struct seven variables. * The seven variables are:. (1) rating the staff organi- 
zation, (2) age of the program, (3), utilization of the Instructional Programming 
Model, 1(4) rating' for facilitative environment, (5) rating of the organization 
of children in the school, (6) use of R&D Center curriculum products, and 
(7) demographic information. These seven variables were .then used to classify 
»the population into strata. 

TKe crossing of different dimensionis, each of which has been segmented-/ 
creates cells containing cases which have received similar values on the 
var;iables inyplved. The use of nonredundant (orthogonal) dimensions provides 
the most parsimonious means of classifying schools into groups that are similar 
with respect to multiple, correlated variables. Since the stratification- ^ 
relevant variables confined redundant information, the variables were orthogona 
ized usijj^g'^principal cJmponents analysis (Harnicin^* 1967) . It was decided that 
64 strata (cells) wa's the largest useful number, resulting, in approximately 
11 schools ^per stratum. This permitted 1;he population to be segmented into 
quart iles along the first principal component and median splits (5^th percen- 
tile) on four oth^r J>rincipal components. - 



The reason fdr-crefti^g'.a^ ^ rame 'wa^ toj^nsur^ that the sample 

would reflect ftili^N^He f ahge of >^ on. thVcharactjeristics measur^ < 

• ^ the' seven^^txjA^l^ from such a "sampling frame 

can be systematic, or it can .be>andcm -j^^ random sampling) . Ta'en- 

sure .that: schooi^.-w^ite^'fe^ from-, the sampling^ frdme -without bias, the selec- 
tion wasj random and 30^ -schools were selected". The sample on^wh^ch^ the evalua- 
^tion wai urlfeimately based, however, was not this /nitiallydrawn sample. . Far 
-^ewer sc^tools than th^ IGE evaluation teaiti had anticipated agreed to partici- 
pate in the evaluation. ^ - ^ V' - * * * 

^ 'T^ere was consequently a danger that, self-selection had created a sample 
that differed significantly from the population in terms of the IGE charac- 
teristics\with which tW evaluation is concerned. In the absence of a care- 
fully developed sampling frame, we would have been unable to determine whether 
the participating schools differed from the population (or.fro^ the initialise 
selected ^sample) , 'The sampling frame, however, provided^a means of assessing' 
whether and Jiow the participant schools differ from the population. Remarkably, 
the self-selected sample appears very sJ&aiiar to the population in terms of 
the stratification variables. (For a more detailed ^account of the sampling 
procedure see Price^ 1977\) 

I ' • • ' ' 

S taff Instrument s 

Q^stionnaires for the sta^f component of- the evaluation .were based pri- 
marily on several existing instruments which were modified in conttrft or ^format 
to meet the requirements of IGE terminology and certain technical constraints, 
such as machine-readable response forms. A detailed diso/ission of the cor^- 
tent and source of each instrument as well a3 -its relationship to the variables 

of interest in Phase I appears in Stewart (1977). ' , ^ 

' 
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' The instriiments were printed* in three questionnaire booklets for distri=* 

bution according to respondent group: Instructional Improvement Committee and 

principal, grade 2 and grdde 5 test units, and all members of the prof ^"i^sional 

staff. One instrument, the Verification Copy of the IGE ,Sclj6ols Question- 

naire, was pifovided separately for the principal alone. Machine-read^able 

response forms were developed to accompany each questionnaire Jpooklet.- A 

• * ^ . 

Jx>oklet, response form(s) , arid* pencil were packed in ari individual envelope 
preprinted with identification numbers and directionsj-f^ distribution, comple- 

tion, and return by the three respondent types, 

, ► ^ » • -» 

The time required for an individual staff member to complete the ques- 

tionnaire_s^varied according to thfe number of respondent groups to which he or 

she belonged. Since the tinit i^eaders of the grade '2 and ^ade 5 test units 

were usuall]f members of all three* respondent groups, they needed to invest the 

greatest amount of time^ — 245 minutes. The estimated working time fgj^-^ach 

booklet was: , : ^ . 

Instflictiorial Improvement Committee Questionnaire r 25 minutes 

General Staff Questionnaire * * " * ' * " * minutes | 

• * - ■■ 

Unit Questionnaire^ . • , 3^20 miritites, 



General staff questionnaires. Every professional staff member assigrt^d 



to the school at least one-third time during both the 1976-77 and 1977-78 
, school years was asked to complete the general questionnaires. ^By setting 



these criteria, it was expe^cted tfiat all support personnel, such as guidance, 
counselors and remeiilal teachers,* would .then partioApate in the evaluation, 
but that staff not 'yet familiar with IGE or not an integral part of ''building 
operations would not do so. Aide^ were .not included );)ecause of the problems 



associated with involving instructional aides .^\it riot clerical aides. 
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• The five questionnaires combined in the single staff booklet and estimated 

• * 

working times were:,;. 

9 

Staff Background Ii^formation ' .20 minutes 

IGE Implementation Survey 30 minutes 

Jo£ Satisfaction Survey -20 minutes f 

Assumptions About Conditions of Effective 

Schooling 15 minutes _^ 

♦» 

Assumptions About Learning and Knowledge 15 minutes , 



r , ' 100 minutes total 

On the Staff Background Information Form (Mendenhall, 1977) each staff 
member provided data on his or her assignment' and position in the school, pro- 
fessional and IGE-related training and experience, professional activities, 
and personal information. - 

Responses to the ^Cj^ Im^ementation Sur^^ey (Klausmeier, 1976) indi^^ed 
each person's judgment of locUl implementation of seven IGE components on a 
five-^int scale from no implementation to ideal implementation. There were 
77 statements or concepts in the survey, each related to one of tKe following 
components: Multiunit Organizational Administrative- Arrangements {HUB) , In- 
struct ionai Programming for the Individual Student (IPM) , Curricular Programs 
(CURR) , Evaluation for Decision-Making (EVAL) , Home-School-Community Relations 
(HSC) , Facilitative Environments (ENV) , and Continuing Research and Development 
(R&D). , ^ • 

The Job Satisfaction Survey (Mendenhall, 1977) measured eight aspects of 
job satisfaction on a five-point scale ranging from not satisfied to very 
satisfied. The eight subscales represented in the 50 items were: co-workers r 
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career function, school identification, financial aspects, worki^ng conditions, 
pupil relations, coiomunit;y relations, and administration/ 

Staff members were asked to indicate the^ relative need for each condition 
listed in a set of 28' statements called Assumptions About Conditions of Effec- 
tive Schooling, T^e statements were derived from IGE literature which defined 
,5^ "desirable conditions foy teaching and learning (Lipham Si Fruth, 1976), Responses 

were on a five-point scale from strongly disag;cee to strong'ly agree , • 
On the last instrument in the General Staff Questionnaire booklet, Assump- 

♦ 

tions About Learning and Knowledge (Barth, 1971) , school personnel Teported their 
feelings about a set of 28 statements related to open edvfcati^. Response 
choices again 'ranged f rpm strongly disagree to strongly agree on a f ive-poirvt 
scale • * 

Unit staff questionnaires . Staff evaluation materials for both Instruction 
and Research (I & R) units, grades 2 and 5, were 'identical. ^However, to insure 
that the two I & R units responded separately, the covers of the unit question- 
naire booklets were printed in different colors, and both the coVers and envelopes 

0 , 

indicated gtade 2 or grs^de 5. , ' 

All members of each I &' R unit, expept aides, were requested 'to respond 

to the questionnaires if they met the following criteria: (1) assignment to 

the "unit at least one^third time in both the 1976-77 ancjr 1977-78 school years, 

and (2) direcjf involvement ih 'planning instruction for unit students in read- 

> 

ins and/or n^thematics and/or language arts. It was assumed that these 
criteria would allow and encourage part-time* unit members such as reading ' 
specialists or'learning disability teachers to respond and yet avoid partici- 
gation by personnel unfamiliar with unit operations. Aides were not included. 

ERIC >^ . . • - - 
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The three instruments in the uni^questionnaire booklet and the estimated 
working times we're: # 

Role of- the Staff Teacher , 10 minutes 

Instructional Practices in Reading ^ \ 

Matheotatics, and Language Arts 60 minutes 



Instruction^ and Researcl>*. Unit Structure 
, and .Function . . 15 minutes 



(• 



85 minutes total 



Each unit staff member responded independent!? ^ to the first two instruments, 
while the ^mit staff as a gr<5up was asked to complete the third instrument. • 
Role of the Staff Teacher (Iro;iside^ 1972) pertained to the number and 

kind of instructional and advisor contact .teachers have with students and ai^p 

^ t • '* 

^ to teaicher specialties.^ Instructional Practices in Reading, Mathematics, and 

** » > * , ' 

Language Arts (adopted by TJ J. Fox from D.eVault^, 1973) 'assessed four aspects 

of individualized instruction — rate, media, grouping^ and learner assessment — 

separately for each of th^-three content areas. 'The Instruction and Research 

Unit Structure and Fiihction Instrument (Ironside, i972) included unit member- 

» * 
ship,, meeting schedules and reports, utilization of meeting time, and the role 

of instrtictional -and clerical, aides . Also incorpprated in this questionnaire 

were questions on topics of special concern in the evaluation: record-keepirtg 

and grouping/regrouping practices, time allocations by curricular area, and 

ij^umber of instructiortal objectives. , • ^ . 

- Instructional Improvement Committee questionrijriVes . If an IIC was operat- * 

ing* in the^ school, members were asked to 'complete by Consensus the Instructional 
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(Program) Improvement Oommittee Structure and Function Questionnaire (Ironside 
- " - . . ■ /- ' 

1972) . Topics in this questionnaire were membership, meeting schedule and 
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reports of meetings, time allocation' to various tasks, and development of 
schoolwide instructional objectives in the various curricular areas. If no 
IXC existed or was operative in the school, the principal was asked to describe 
in brief narrative fobt the governing: body or leadership group or person and 

to complete the section of ^ schoolwide objectiv^sl ' The IXC booklet also 

• ' . m ^ ' . ' 

included a question regarding the demographic category of the school attendance 

area. Categories were*ased on the pomiriunity types developed by the National 

Assessment of Educational Progress projects ' - ' 

The verification copy of the I GE. Schools Questiohiiai^e -fwas distributed 

with the XXC materials. Principals were asked to, verify. or update the infor- 

mation previously proyided; of particul^i? imp6rtance were changes* in the 

schools' organization and extent of application bf the .Instructional Pro- 



grairanmg Model, for example, addition of another subject area in which the IPM 

is«applied.^ The estimated time required for* completing' both IlC/principal 

^ ' * » ' 

^ 

questionnaires was 25 minutes. ' *. . ; • . ^ . 

student Instruments ^ ^ ^ * , . * - 

Across the 162 Phase I schools from which stuSpnt data were received, 
approximately ^5, 500 students at grade 2 And S,800 students at grade 5 were * 
tested: Most schools tested between 24 and^ 50 ^tudents at each grade. -Th*ese 
\students were preassigned by the R&D Center to one of four test groups (A> B, 

t, or D) m: each graded Also preassigned were the tests to. be ^ administered 

1 ' ' ' ' * * 

to gach group, ^he Short Form Test of Academic Aptitude (SFJAA) .was the only 

test administered to all groups. Since the total tim^' requirement was approxi- 

mately 90 minutes for grade 2 students and approximately 120 minutes for 

grade 5 students, the tests were^ scheduled to be administered in three 

separate sessions. • Og . _ 



Listed in Table 1 are th^; tests administered at both grades. Test copies 
of the SFTAA, California Test of Basic Skills (CTBSj and Self -Observation 
Scaleg (SOS), were purchased from the commercial publisher; machine-scorable 

^§FTAi^,and CTBS test books were used at grade 2, while reusable test books for. 

"^hese two measures wer^ used with a combination SFTAA-CTBS answer sheet for 
grade 5. Since each'' CTBS. Reading and Mathematics test was assigned to a spe- 

- cific group, students used only a portion of a test book or answer sheet. 

The SOS is a self-report, group-administered instrument with empirically 

determined scales^ which measure the way children perceive themselves and their 

relationships to peers, teacheit, and school. The Primary Level measures four, 

dimensions of children's self concept: 'self-acceptance, social maturity, 

school affiliation, ^nd self-security. Seven dimensions are measured With 

the Intermediate Level: self-acceptance, self-security, social maturity, 

social confidence, schoal affiliation, teacher affiliation, and peer affilia- 

* * 

tion. . * ' . , ^ 

The five Concept Attainment Abilities (CAA) tests are trom a battery 'of 
tests that was developed as a part of a* previously completed Center project 

% 

(Harrises Hkrris, 1973). Only students in grade 5 participated in thaf study^ ^ 

The CAA tests were administered to assess student cQgnitive skills in tjiree 

cat^ories — numerical ability, memory, and word, fluency . The tests correspond 

to these categories as follows: » ^ 

•» • • 

Cognitive Ability ^ ^ ' Test 

Numerical ' * , ^ Number Series 

* Number Relations 
Memory* / Picture Class Membry 

s ^ , Remembering Classes: Members 



Table 1 
Phase I Student Tests 



Test 

a 

Short Form Test of Academic Aptitude (SFTAA) 

b 

Comprehensive Tests of Basic Skills (CTBS) , Form S 

Reading Vocabulary 

heading Comprehension; Sentences 

Reading Comprehension: Passages 

Reading Comprehension , 

Mathematics Computation 

Mathematics Concepts & Applications 

Spelling 



Self Observation Scales' (SOS),^ Form C^ 



e 

Locus of Control ' (Cromwell, 1964) 



Concept Attainment Abilities (CAA) 

Niamber Series 
Niamber Relations 
Picture Class Memory 
Remembering Classes: Members 
Omelet^ 



^Grade 2, Level* 1? Grade 5, Level 3.-^ 

Grade 2, Level C? Grade 5, Level 2. 
^Administered as a CAA word, fluency substitute'. 
^Grade 2, 'Primary Level; Grade 5, Intermediate Level.' 
^Not included in the analysis because of low reliability. 
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Word fluency Omelet (an anagrams test) 

Spelling (CTOS) 

Since comparable tests are not available for grade 2, the CAA tests were given 
only at grade 5 for the Phase I evaluation. 

o 

' Measures of School and X SV-R Unit Characteristics 

' The structural 'model was formed before tjie variables used in the model 
were created, ^lowever-, vthe model was formed with knowledge of the inf oration 
from which the variables would bfe created, the initial model was developed by 
the evaluation^taf f to satisfy the following conditions: (a) to be a fair 

^rendering of the IGE model; tb) to be' specifiable in terms of the item ^ool;* 
and (c) to have only "main effect" .variables (i.e., no attempt was made to 
address interactive relations among variables) . 

First, all items from all nonstudent instruments weije grouped together. 
Second, the evaluative staff independently divided the items" into sets of 
items each representative of a single variable. Having, followed this procedure 
independently, members of the group then met and reached consensus on groups of 
items that defined a particular variable. . After this procedure for selecting item 
sets, each group of items was given a verbal description that reflected the 
information contained in its constituents.'' Next the selected items. in*'each 
group were combined to form a composite variable. 4 ^ » 

0 To appraise how well a model fits data, one must begin with trustworthy 
data. A weakness of the Phase I data stems from the fact that already 
existing questionnaires were, used. Had there been resources arid^time, it 
clearly Jwould have been preferable to.buiid questionnaire items specifically - 

--to measure the constructs of an a priori model. As it was, available 

■41 



items were used* to scale.' the variables. For several\ariables, however, there 
were many pertinent items available, so, for those variables, little damage 
-was done by relying upon' available questionnaires, 

A second limitation of the data stems from, the remoteness of the data 
collection procedure and th^ amount of. time it took school staff members to 
respond to the questionnaires. This questionnaire form of data collection 
invytes hurried responding. Moreover, respohdents may have wanted to respond 
as tifey thought good IGE citizens should, since they knew that th? responses 
would be sent back- to the Wisconsin Research & Development Center, There is 
no doubt that the signal o»e receives comes partly veiled With noise when one . 
uses mailed quest ioniraires. . • 

The evaluation staff was aware^of these limitations from the outset 
and took steps to minimize their effect. An innovative use of questionnaire 
responses was developed which deserves" mention, pually, questionnaire items, 
like test items, are combined in a linear manner to form a scale. That was-, 

v*^ ^ * \ ' * * * 

not always done in our'study. Rather, Boolean logical expressions were often 
used to combine the responses 'on several items into nfew, composite items to be 
arranged as a scale, ^'or measurement purposes, these composites were not them- 
selves present in any qiiestionnaire. They were the product of logical opera- 
tions performed on questionnaire items. 

There were two reasons for taking this approach to scalirvg. One was to 
"goof-proof" our variables*.' The detection of contradictions and other con- 
vergent uses of questionnaire responses, were used to ^minimize the extent to 
which our scaled values could be thrown off by erroneous responses. Some forms 
of deliberate distortion were anticipated, and scaling decisions were made to 
minimize their effect. A seconci reason for* the approach Tiad nothing* to do 

.'}2 
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with accuracy of responses. In some cases we decided that, even if we assumed 
the responses were perfectly accurate, a justifiable ordering would not be ob- 
tained by arithmetic combination of item responses. 

As the report from a field validation study (Ironside & Conaway, 1979) 
showed, there was ample reason for attempting to "goof -proof" the variables. 
In that ^tudy, Phase II of the- evaluation^, which involved 30 of the Phase I 
schools, a field check was run on the validity of questionnaire responses. 
Correlations^, between Phase II field ratings and thase I questionnaire-derived 
values are available for some variables: 

Interorganizational Relations ^ ^ .66 

Procedures Fostering Coordination and 

Improvement of the School Program ^ ,53 ' 

ft t ' 

Intraorganizational 'Structure , •*.60 

General Implementation of the IPM| ' ,67 

Those variables, concerned as they are were surface organizational features 
of IGE schools, were probably measured better than the instructional practices,. 
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^ Description of variablps 

Eight of the variables used in this study -measure aspects of the school 

as a whole ^ Such a^ organizational features^ schoolwide practices, staff back- 

ground, o^itKjgraphy, and teacher job satisfaction. Other variables do not per- 

tain to the, school as .a whole; they are measures of a>fepecific I & R unit's 

practices In^-^eading and in matK.^ Each variable specific to an I & R unit is 

actually treated as four separate variables. It is measured in two I & R units 
* / 

per school — dne that includes children of grade 2 age and one that includes - 
cjAldren of grade 5 age. In each I & R unit, there is a reading version And 
a math version of the variable.*' The schoolwide variables are described fi|:st. 
Variable^ specific to the I 6 R unit are described thereafter. ' • 

Schoolwide Variables 



"^even variables describe gerferal characteristics of schools and their 
st^fs. Three of the schoolwide variables, Interorganizational Relations (lOR) 
Intraorganizational Structure (lOS), and Procedures Fostering Coordination and 

Improvement^p^ the School Program (GOS) , axe measures of organizational fea- 

^ * ' • . \ ^ ' , 

turesc These orgarrizational features are supposed to foster staff job satis- 

I * • • ' 

faction and ^effective instructional prferctices. Two other schoolwide variables, 

• • , . _ ' . . ' •/ 

General Staff Background (^p) ' and Belief in Individ.ual Differences (INDIV) , 

are measure^ of teachers' preparation, experience, and beliefs. These charac- 

'teri'stics should also be relat{ed to job satisfaction and the use of effective 

instructional practices. Our s-Jimctural model makes no •formal attempt to 

enumerate ui^derlying factors that influence 'differences among schools on th§ 
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five variables itiention.ed thus far. For the purposes, of the Phase I study, dif- 
ferences among schools on these 'variables are taken as given* Accordingiy , 
Figure 2 shows no straight arrows ppinting toward these' five .variables*. In 
the technical vocaburary of structural equations methods, such variables are 
called exogenous Variables* ^ . 

A sixth general s<?hoplwide' variable is Sghoolwide Implementat^ion of the * 
Instructional P;:pgrammi'ng Model (SIPM) . Differences amoilg' schools on this 
yariable^are thought to fcfe affected by four of^the exogenous- variables (namely, 
(JOS, lOS, GSB, and INDIV) . From the point o^ view of IGE imf)lQmenters, IGE's 

4 . V ' OS* • 

distinctive organizational structures, schoolwide procedures, and interorgani- 
' ' « * • 

,zational relations are mecint to facilitate implementation^, of the Instructional 

Programming ^odel. SIPM is not an exogenous varieible, because our model ^ ^ 

enumerates expected causes of .its variations.^ Variables wYioge determinants are 

specified by a models like SIPM, are referred to as endogenous variables * 

A seventh schoolwide variable is Teacher Job Satisfaction- ( JOBSAT) . * Job 

satisfaction — itse'lf an^outcome of iht^rfest — is kept out of tl^e analysis that 

deals with student achievement/ Results concerning JOBSAT a^e* given later in 

the repprt. ^ ' • * ' . . 

.The eighth variable. School's Demographic Setting (DB — standing for demo- 

graphic background), describes the * cominunp.ty and student population served by 

« 

a school. It is not supposed to be related to the other schoolwide variabl^^,, 

nor s^uld it be related to general agpectg of instructional practiced in the 

I & R xinit. DB has been included because of its well known corj;ela,tion with 

, • « 

. ft 

student achievement. ' * .^^ 



In the following patar^graphs, eaph of, these schoolwide variables is given 
a verbal definition. A description of each variable's distribution and in- 
ternal consistency is also given/ Readers wanting to know more eibout a 



specific variable should refer to the detailed technical report which corres- 

* ' ' ' 

ponds to that variable; Table 2 lists technical reports and the' variables with 

,j * * 

* '*'<.' ^ ' ' 

which they correspond. Table 3 presen^ts summary statistics for these variables. 

. ' ,( ^ ' ^ 

The relation 'of internal .consistency to reliab^ity is such that a com- 

»> 

posite whose constituen^ts have^hicjh Internal consistency may, pr mlay not possess 
high test-retest reliability. Conversely, a composite that has high test- 
retest .reliability may have low internal consistency,. Any composite that 
measures a hqmog*eneous construct should be expected to have high internal* con- 
si^ency.<?* However, a composite that metasures a construct combining several 

yempij^^ically unrelated facets v^iild not necessarily" have high internal consis- 
tency, as noted by Cronbach and Meehl J[1955) . Several of the compo^itejs used. 

"in this, study fit the latter description, and, accordingly, should not be ex- 
pected to have high internal consistency, ^ ^ 

• ' f ■' ' ' ' . 

Int^rorganizational Relations (lOR) . lOR is a measure of the school's 

f > ^ 4 

r^elationships and^ activities with outside persons and orgcinizations, especially 

relationships and activities bejieved to facilitate implementing^ and maintaining 
IGE in the school. lOR includes ^traorganizational arrangemer^ts and activities 
of - the .school and dts staff members that kSep the school informed o-f community 
needs, and new educational ideas. 

. Intraorganizational Structure (IPS) . lOS is a measure of certain aspects^ 
of the school's internal organization that are 'relevant to implementing IGE. y 
Organizational structures within the/school (Instructional Improvement Com- 
nfittee, Instruction & Research luiits, etc. X 'are ^si^essed for characteristrcs 



Table 2 



Technical Reports Describing SchooV Variables 



Used m Phase I Study 



School wide Variables 



Technical Report No* 



Interorganizatipnal Relations (lOR) \ 

Procedures Fostering Coordination and 

Improvement of the School Program (GOS) 

Ggrieral Staff Background (GSB) 

Intraorganizational Structure - 

Belief in Individual Differences (INDIV) 

School's Demographic Setting (DB) 
\ » • 
^Schoolwide Implementation of the 

Instructional Programming Model (SIPM) 

Teacher Job Satisfaction (JOBSAT) ^ 
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477 
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480 



482 



483 

a b 
484 ^ 512 



Rripe, Janicki, Howard, Stewart, Buchcinan, &. Romberg {1978a -»^1978h)* 



Price, franicki, VanDeven€er, & Rom3;>erg (1979) 
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Table 3 

Sui^ary Statistijrs for Schoolwide .Variables 



1 






• 








4 




Variable 


Mean 


S.D; 


Median 


^ Actual 
^tiinimum 


Logical 
Minimum 


. Actual 
Maximum 


f 

Logical 
Maximum 


o( 


lOR 


20.486 


6.809 


20.228 


I 6.286 


0 


3^.812 \ 


42 


.64 


lOS^ 


20.522 


3.802 


21.223 


8.092 


'1.50 




33.3 


.63 


GOS 


58 .178 


9.908 


57.934 


. 27^083 ' 


0> ^ ' 


B3.6B2 


104 


.67 


GSB J 


3.844 


.618 ' 


3.8,96 


• 1.884 


0 


5.021 , 


7.0 


■ .26 


INDIV 

4 


3,163 ' 


.179 


3.152 


f 2. '5 5 6 


* 0 


3.667 


4.0 


.87 


^IPM 


''fc.441 


0^2.263 


' 64.061 


^6.578 


0 


t 93.446 


100 


. .87' 




4.096-^* 


2:.50'2 


. 4.2% 

» 0 




1' 


7 ' 


7 


a 


JOBSAai , 


40.571 




41.^001 

< - ♦ 

0 


24.600/ 

■ 


V 

0 


50.000 


50 • 


.94. 



PDB is a one item variable; therefore ncr irftemal consistency carv be calculated. 
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such as membership composition, frequency of meetings, permanence of leader- 
ship, amount of release time made available for meetings, whether parents and 
others participate . in the activities of such groups, whether agenda of meetings 
are kept, and how agenda afe distributed. The. existence and responsibilities^ 
of certain supplementary staff positions (IMC directoa^s, student teachers, 
aides, and interns) are al3o assesseci as part of the internal organization 

of the school.- ^ ^ ^ 

« 

. Procedures Fos^tering Coordination ^nd Improvf^ent of the School. Prograirr ^ 
^{GOS)_. ^GOS is a measure of procedure^*in\the school that are supposed to' 
f<?ster^ continuing improvement dt the overall school program. Included are 
research and development, staff development, use of volunteers ^nd aides,' 
noninstructional (advisory) contact between teachers and students, and other 
aspects of home-schoolT-community relations.' We iregret that the abbreviated 
name of this measure, GOS, has little mnemonic value. "GOS" stems from an' 
earlier name by whicfi" we knew this variable — retained for the benefit of 
readers who will go on to read Technical Report 477 (Price et al., 1978b) where- 

' r r , 

"GOS" appears in many computing statements. 

General Staff Background (gSB) . GSB is a measure of teaching experience 
in IGE, teaching experience in general, and leadership activities in the pro- 
fession. ^ . • ' 

Belief in Individual Differences (INDIV) . INDI\r is a measure of teachers* 
belief that students have individual needs which should be considered in,*plan- 
ping and implementing an instructional program. * 

y ^ 

General^ Schoolwide Implementation of -the Instructional Programming Model 

' ' f 

(gIPM) . SIPM is a measure of implementation of general school .practices^that 
nav^ been encouraged by the Wisconsin R&6 Center as supportive of the Instruc- 
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tional Programming Mo<3el (IPM) ♦ SIPM is based, on s^^-reported practices of: 
(a) setting schoolwide .^o^nstruc tional objectives by the Instructional Improve- 
ment Committee ClfC) ; (b) adapting ^schoolwide objectives in Instruction & Re- 
search (I & R) units? (c) tising IIC gu^^ance in the developttent of recbrd- 
keeping procedures; and (d) providing for carrying out the IPM in the I & R 
■units of the school* 

School's Demographic Setting (DB) . DB is an ordinal demographic classi- 
fication of the community served by a schopl*. As such/ it is a proxy for ex- 
periences, skills, and attitudes that students acquire putsi^e the school, 
but which affect outcomes of interest to educators. This measure classifies 
the community of a s.chool into 6ne of the seven demographic categories em- 
ployed in the National Assessment of Educational Progress; the categories 
are arranged in order o^ the rank they had on^ student achj.evement in the 
National Assessment, DB was given consideration as a covariate for analyses 
involving student achievement. It turned out, however, ^hat DB was negligibly 
correlated witTi measures of organizational features* and school practices. Its 

multiple correlation with thfe six schoolwide varictbles just described was .197, 
« * » 

which is not statistically significcUit [F'(6,149) = 1.01, p > .40]. Its/effect 

* / ^ 

as a. covariate was nil, *so it was not. employed as a covariate in the ^alyses 
reported here. ' ^ ^ 

Teacher Job Satisfaction (JOBSSt) i JOBSAT is a schoolwide measure of 
teachers' satisfaction with a variety of aspects concerning their school and * 

V 

professions. It was based on the SO-item questionnaire that is contained in 

■ • ^ ' / ' . " - 

Technical Reportir512 l^rice, Janicki, VanDeventer, & Ronfcerg, 1979). 
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Measures of I & ,R Units' Practices in Reading and in-Math 

There are four variables that measure aspects of instruction in a specir 

fic I & R lanit, rather than in the school^s a whple. These variables are^ 

Utilization of Curriculum^ Resources (UCR) , InformatjLon Acquisition (lA), 

Individualization of Instructional Decisions (IDM),\and Management of Grouping 

and Instructional Continuity (lEy., The practices of\an I & R unit can diffe?*' 
* ^ ^ \ 

from one curriculum area * to another. Consequently, tW^se variables have been 

measured separately in two curriculum areas, reading and math, which are the 

areas for which measures of student achievement have been obtained in' this 

study. These four variables are intended to measure processes that are closer 

'to what pupils actually experience than the schoolwide Variables—processes 

which are supposed to mediate the Influence of schoolwide variables. ^ 

Two I & R units ,.^ora each schbol are inclu4ed in the* study- yone that in- 

f . ^ , * 

eludes children of grade". 2 age and one that includes children of grade 5 age. 

m \ 

Each measure of I & R unit practices may thus be thought of as having four 

^ ' * ^ 

versions: practices of ^he grade 2 I & R unit in reading, the^ade 2 I & R 

unit in math, the grade 5 I & R unit in reading, and the grade 5 I & R unit in 

math. 

In the following paragraphs, each measure of 5 & R unit practices is*- 
given a verbal definition. Readers wanting to know more about a specif ic;^,, 
variable shouPd refer to the de'tailed technical report which corresponds to 
* €hat variable. Table 4 lists' technical reports and the variables to which they 
cbrrespond. Summary statistics for tlil four versions^ of each variable are 
given in TarflLe .5- • ; - ^ ' 
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Table 4 

Technical Reports Describing Unit* 
Variables Used in Phase I Study 



I & R Unit Speciific Variable' Technical Report' No. ^ 



Utilization of Curriculiam Resources (UCR) , 485 

Information Acquisition (lA) ' 486 

Individualization* of Instructional 



Decisions (IDM) j 431 
Management of Grouping and of 



Instructional Continuity (IE) 



488 



a 

Price, Janicki, Howard/ Stewart, Buchcincui, & Rgjgberg (19781 



19781) . 
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Summary Statistics for Unit- Variables 








9 

Version • 




Mean 


SD 


Actual 
Median min imum 


Logical 

minimum 
^ 1 


civ* L«UCU> 

maximum 


T.ncr i pa 1 

maximum 












Utilization 


of Curriculum Resources (UCR) 




* 




Grade 


2, 


Math. 


151 


* /■ 
39*528 


9.611 


39.500 11.944 


0 


y 60 


60 


' .29 


Grade 


2, 


Reading 


156 . 


44* §45 


9.587 


46. 453' 22.000 


0 


60 


60 


.27 


Grade 


s, 


Math 


151 


40.i48' 


9.530 


40.000 12.000 


e 

0 


60^ 


-.60 


.20 


Grade 


s, 


Reading ' 


156 


44.770 


, 9.890 

r 


,44.667 19.910 


0 


60 




.35 


Information Acquisition (lA) ' • . ^ - 


Grade 


2, 


r 

Math 


151 


1.176 


1.276 


8.00Cf • 4.0 


1 


8.0 


8 


.00 


Grade 


2, 


Reading 


156 


7.0t)5 


.997 


7.001 ' 4.0 


-1 


8.0 


8 


.04. 


Grade 


s, 


Math 


151 


7.386 


1.155- 


8.000 / 3,0 ' 


1 


8.0 ' 


8* 


.07 


Grade 


s, 


Reading 


^156 


6.928 


1.015 

• * 


7.000 4.5 ' 




8.0 * 


8 


,.00 
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Table 5 (continued) 















, Median 


Actual 
minimum 


Logical 


Actual' 


Logical 




Version 


N 


Mectp 




minimum - 


maximum 


maximum 










\ 

Individualization 


of Instructional Decisions, (IDM) 








Grade 


2, 


, — 

Math 


151^ 


8.332 


2.605 


8.000 


1.000 


0 


14.00 


14 


-.02 


Grade 


2, 


Reading 


156 


8.158 


2.379 


8.223 


2.333 


0 


13.00 


14 


.08 


VxL ClUC 




i iCX I- 11 


151 


' 8.092 


2.553* 


8.000 


1.000 . ^ 


,0 


14.00 


4 

14 


' .10 


Grade 


5, 


Reading 


156 


7.511 


2.181 


7.125 


1.500 ' 


0 


13.00 


14 


' :o4 








Management of Grouping and of Instructional Continuity 


(IE) 






Grade 


2, 


Math 


151 


29.581 


7.258 


30.000 


14.000 


0 


45^750 


48 


.19 


Grade 


2, 


Reading 


156 


28.638 


6.120 


28.338 


« 

13.668 


0 


45.000 


48 


.58 


Grade 


5, 


Math 


151 


28.536 


7.805 


28. -250 * 


12.^000 


Q 


45. 000 


48 


.22 


Grade 


f/ 


Reading 


156 . 


26.339 
> 


5.907 


26.000 


1^.000* 


0 


42.875 


• 48 . 


.63 

9p 
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U'tilization of Curriculum Resources (UCR) , UCR is a measure of the extent 
to which the I & R unit uses a variety of IPM-compatible curricular materials 
and media iVi a. particular subject area (reading, math). 

Informatibn Acquisition (lA) . lA is a measure of: (a)' the completeness i 

• J 

of information gathered for making decisions about .the instruction of individual 
students; (b) the variety of means for gathering such information; and (c) the 
use of record keeping procedures by the I & R unit for organizing and retrieving 
the information. The measure is specific to subject area and to each I •& R 
unit. ^ 

Individualization of Instructional Decisions (IDM) > IDM is a measure of 

-, i * 

the" extent to which decisions affecting instruction ar6 adapted to differences 

among students. This is made evident by the basis used for moving children to 
new ^material , the different rates of individual firogre.ss through curricular 
units, and the percentage of I & R unit meeting time spent making instruc- 
tional decisions. 

Management of Grouping and of Instructional Continuity (IE) . IE is 
a measure of several aspects of environmental organizations. This variable 
includes arrangement qf the instructional environment in terms of scheduling, 

"I 

group size and change patterns, and opportunities for children to become 



aware of interrelationships and continuity in instruction. 

Measures of Student Achievement 

It should be noted that the final analysis of pupil data involved only 
the achievement measures for reading and mathematics. Data on student cogni- 
tive abilities and personality development had been gathered, but were not 
used in the analysis. 

~57. 
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, Grade 2 . The Comprehensive Tests of feasic Skills (CTBS), Form S, Level ^, 
Expanded Edition, provided measures of reading and mathematics achievement in 
. Grade 2. The following subtests of the CTBS were administered: reading vo- 
cabulary, reading comprehension of sentences, reading comprehension of pas- 
sages, mathematics computation, and mathematics concepts and applications. Be- 
cause of time constraints, most students took only some of the CTBS subtests. 
It -was unnecessary to require each student to take every test,- since the ' 
grade 2 instruction and research (I & R) unit, rather than the individual 
student, was the, unit of analysis. CTBS subtest scores of indi<vidual pupils 
in each I & R unit were averaged to form an aggregate measure of I & R 
unit. Prior to that aggregation, however, adjustments were made to individuals 
subtest scores. Those adjustments are described in the next section of this 
report and, in more detail/- in Techriical Report 408 (Price, VanDeventer, 
Janicki, & Romberg, 1979,) 

Grade 5 , The same general approach taken in grade 2 I & R units to form 
overall reading achievement and mathematics achievement scores was also tak-en 
in grade 5 I & R un4-ts, A different level of achievement test was used, how- 
ever, and that test included some additional subtests. In grade 5, the Com- 

^ t ' -J 0 

prehensive Test? of Basic Skills (PTBS) , Form S, Level 2, provided measures of 
reading and mathematics achievement. The following subtests werel administered: 

reading vocabulairy, 'Reading comprehension, ^mathematics .computation, and mathe- 

\ ' * 

matics concepts and applications. More detail about grade 5 achievement scores 

and their adjustment was given in Technical Report 509 (Price, VanDeventer, 

-Janicki, & Romberg, 1979h) . 
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Adjustment and Aggregation of Student Achievement Scores 

Literature promoting IGE suggests that student achievement will be raisefl 
by implementing the features of IGE (e.g., Klausmeier, 1977; Klausmeier, 

\ * 

Karges, & Krupa, 1977). Therefore, student achievement measures are an 

important part of the Phase I evaluation. One would expect schools high on 

implementing IGE features to have high achieving students. Examining the . 

relations between IGE features and student achievement, however, can be mis- 

leading**if 'differences due' tcy other influences are^ not accounted for. Students 

from a school in a wealthy suburb, for example, would commonly score higher on 

an achievement test than students from a school in a poor urban neighborhood 

(Coleman, Campbell, Hobson, McPartland, Mood, Weinfeld, & York., 1966) • Some 

differences in academic achievement between rich and poor children ^ippear to 

persist even when their school programs are identical (Firkbwska, Ostrowska, 

Soskolowska, Stein, Susseis^ & Wald, 1978). Achievement differences associated 

with community type cannot be ignofred. Presumably, a major reason why students 

in rich and poor neighborhood schools would differ in achievement is that they 

are unequal at the outset in their command of the knowledge and skiUs required 

for sucfcessful school performamce. 

Schools are unevenly matched in terms of the student populations they 

serve. That fact is conanonly recognized when schools are dompared^directly. 

Teachers and administrators from. different schools are usually aware of the 

disparate backgrounds f^|^ which each other's students C9me. Howfever, th^^ 

need to control for differences in student background less obvious when 

relationships between school characteristics and student achievement are being 
* . • / 

studied. ^The influence of a school characfteristic or practice on achievement 

' • ■ ' 59 ■ ■ ' 
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can be obscured if differences inS^dent background are not controlled. An * 
educational practice that has no^ effect on achievement can be correlated wi-th 
achievement if most schoijls using the practice serve a population of education- 
ally advantaged students. Conversely, an effective practice can appear inef fec- 
tlve if most schools using it serve a population of educationally disadvantaged 
students. • ^ 

To control for differences in student achievement that result from dif- 
ferences in^ student populations, jve obtained two measures: the demographic 

^ and economic character of each school's comiu.unity (demographic -background) , 
and student's academic ability. 

f Demographic background pertains to the school neighborhood as a whole. 

Consequently, it is not sensitive to educationally important differences that 
exist between^ the backgrounds of children who attend the same sfchool. It 
simply represents the kind of neighbor^hood being 3ervfe<^ suburban, inner city, 
small town, rural, etc. As such, demographic background was seen as^ a reason- 
able proxy for the schoaj. setting, the educational ambience of the community, 

V 

and the prevailing economic circumstances of students' families. As mentioned 
earlier, the school demographic background measure proved to be negligibly re- 
lated to other school characteristics measured. It was correlated with student 
achievement scores (from .29 with Mathematics achievement in grade 2 to .39 
with Reading achievement in grade 5) , but its correlcition was less once adjust- 
i.»^nl w«i.'. mailt* 1 or individual :>tudenL;;' scores on tlic SiiurL Form Test of Aca- 
demic Aptitude. 

Th^ Short Form Test Academic Aptitude (SFTAA) , a measure of scholas- 
tically useful skills and knowledge, was given to each child. This measure 
provided information about individual students* short-term prospects for 

o 

success in school .subjects. A high score on the SFTAA suggests, that a student 
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has acquired'r-t;^ough home, prior schoolingr and elsewhere— -skills and knowledge 

that make it easier 'to succeed in school. Since there are innumerable ways to 

get 'a low score on tests such as the SFTAA, the educational prospecits of low 

scoring students are less clear^ Nevertheless, a low score suggests that a 

student may lack skills and knowledge useful to school learning. The SFTAA, 

therefore, gives probabilistic information ab9ut scholastically useful skills 

and knowledge of students. Like any probabilistic information, it can be 

wrong in individual cases. , 

The academic ability scores of individual students can be used to pre- 

diet how those students will fare -on subject area achievement 'tests. That 

prediction, in fact, was our primary means of making adjustments ,for achieve- 

ment test diffejrencej^ that 'could not be reasonably attributed to school program 

differences. However, prediction on the basis of informaticsn specific to the 

* ^ ... 

individual student reveals only part of the effect academic ability can have 

on achievement test performance. By aggregating the SFTAA scores of students 

I 

ill a school, a measure of the school's educational milieu is provided. This 
can augment the prediction provided by individual scores. In other words, 

one might predict different achievement from two students who obtained the 

• *« 
same SFTAA score, if one tattended a school in whidh most students, scored low 

on the SFTAA and the other attended ^a school in which most scored high. 

Student raw scores on achievement measures were carefully adjusted to 

provide a better comparison among schools. The student achievement variables 

used in subsequent analyses of the Phase I study were not simply raw scores, 

but residual scores derived from the differences between raw and predictod 

scores on achievement tests. Each student • s^^cores on academic abijLity tests 

were used to predict his or her scores on the achievement tests. The student's 

actual scores on those measures were then pompared with the predictions. The 
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deviations of the actual scores from the predicted scores were the variables 
Qf interest. These difference scores, knovm technically as regression residual 

were t4iQn aggregated • ' . *' 

• < 

Guarding Against Faults 'in SFTAA Scores 

The adjusted achievement scores, central to subsequent analyses, are sus- 
ceptible to any inaccuracies thal^ might exist in SFTAA scores. An unrepresenta 
tive score on the SFTAA would cause an inaplpropriate adjustment to be made on 
a student's achievement subtest scpre^s. Therefore, special pains were taken 
to assure that^ this basis for adjustment had values which seemed reasonable^ 

One cause of unrepresentative SFTAA scores 'i^ mechanical err^s in taking 
the test. Second grade pupils, 'who have little experience in taking paper-and- 
p^cil tests ,^ are likely to make such errors. / 
V Faulty test administration is another possible source of difficulty. If 
a teacher's explanations qr directions are unclear, student scores can be ad- 
versely affected. Some questions on the SFTAA are read to the students by 

the teacher, so some teacher-caused problem^ are inevitable. Errors in timing 

« 

the tests could distort the scares considerably. Finally, '.a student might not 

' ' . * 

achieve a representative score because of a bad day, honte troubles, illness, 
test apxiety, discomfort about the testing room, or distractions in a group 
setting. 

Winsorization. A procedure known as "winsorization" was used,^o help 
avoid problems that can beset a° one-time measure of academic ability. 
Generally, winsorization involves replacing extreme scores with scores that 
are still on the outskirts of a distribution, but^hich are less extreme (see 
Anscombe, I960; Dixon & Tukey, 1968). In effect, 'winsorization pulls improb- 

e2\ 
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ably high or low scores back to a believable range. Unrepresentative low ' 
scores on the SFTAA occur much more frequently than unrepresentative ^igh 
scores, so only the lower tail of the distribution was winsorized for thi-s 
study. 

Thus far, we have described winsorization as a technique used only to* ad- 
just extreme values in a univariate distribution — th^t is, values of a particu- 
lar variable that are so extreme as to be implausible. In 'addition, winsoriza- 
tion can be applied to outliers from a bivariate distribution . Implausible 
pair-wise combinations of values are s^ingled^ out for winsorization in a bi- 
variate distribution. 

r 

Winsorization with respect to bivariate distribution was the approach 
used in the present study. The two variables, whose joint distribution was • 
adjusted for outliers, are raw scores from the %wo halves of the SFTAA: 
language ability and nonlanguage ability. These subtests are correlated, 
but it is common for individuals to have, a higher normative standing on one 
subtest than on the other. It is presently fashionable in educational prac- 
tice to interpret a large discrepancy between such scores as symptomatic of 
problems that can interfere with school learning. However, a test that is 
susceptible to misrepresenting a child's underlying competence is a poor basis 
for making inferences about such problems. Consequently , /we make no psycho- 
educational interpretation of discrepancies between children* s. language 
ability and nonlapguage ability scores. 

Some children will have authentically large differences between scores 
on two tests, yet for a particular population of children there are bounds 
within which discrepcuicies must stay to be credible. Discrepancies beyond 
those bounds can be caused by measurement procedures that have gone awry. 



63 



54 



Alternatively, they can accurately describe a child who is so atypical that 

« 

the evaluator should question whether it is taxoncxnically correct to include 
the child in the same population as the other children* Children who are very 
atypical should be "removed from an analysis to avoid a distorted picture of the 
population* If there is insufficient evidence to justify such removal then 
winsorizHtion is appropriate* Winsorization does not eliminate large dis- 
crepancies, but it reduces them to a plausible level. 

ks mentioned j>reviously, unrepresentatively low scores were more likely 
to occur than unrepiresentatively high scores. Therefore, when the discrep- 
ancy between language ability and nonlanguage 'ability was implausibly large, 
the lower of the two scores was winsorized. ' The higher score was not; 

V 

Based* on the bivariate distribution, 95% confidence , bands for individual 



scores 



were calculated for language ability and nonlanguage ability, ^;espec- 



tively. To make these calculations, each variable had to be regressed on ^ the 
other. Scores falling too' far below one of the regression lines were adjusted 
back to the lower bound of the confidence interval for that. regression line.y| 

In practice , students with low scores on both language and nonlanguage 
ability, or with two high scores, did not- have their scores adjusted. Only 
students with relatively good scores on one subtest and very poor scores on 
the other received adjusted scores. By adjusting only as far as the coofi- 
dence band/ t^e distortion of erroneous outliers was minimized. Out of 5,352 
students, only the extremely discrepant scores of 242 students were affected 
by the winsorization procedure. 

Adjusted Achievement Scores as Regression Residuals - 

The SFTAA language ability and nonlanguage ability measures .were then 
used as covariates to adjust the CTBS achievement subtest scores of each ^ 
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student. Scores of all students who took an . achievement: subtest (N approxi- 

I 

. mately 1,384) were used in the regression analysis. Each, CTBS subtest (in 
raw score form) we^ regressed on SFTAA language < aJiility and nonlanguage 
ability (also in raw score "^orm) and on sex. 

Each student's predicted, score on each subtest was estimated from the 
regression equation. The predicted score was then subtracted from the 
student's actual raw score on the subtest, thereby creating a- residual score 
for each subtest. This residual score reflects the s^ddnt's achievement 
after being statistically adjusted to cdntrol for the effects of academic ^ - 
ability and sex. 

Aggregation of Adjusted Achievement Scores " 

The r^^^iduai scores of* individual students on CTBS subtests provided the 

basis for 'an aggregated measure representing eacfi school. The subtest:^' are, 
* • - 

in the case of grade 2, reading* vocabulary, reading comprehension of sentences, 
reading comprehension of passages, mathematics computation, and mathematics 
concepts and applications. The subtests for grade 5 are, in sequence, reading 
vocabulary, reading comprehension, mathematics computation, mathematics concepts, 
and mathematic^ applications. The scores of each group of students in each 
school were averaged to form a school mean for each subtest. In other words. 



Performance on the SFTAA can be scaled in several ways. We .used raw 
score scaling in preference to IQ scaling. ^When an ability test such as the 
SFTAA is being used to predict school achievement, * its IQ scale form can yield 
unintended results unless age is also included as a pred^ictor. 
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the adjusted scores (regression residuals) of -students in an I & R lanit were 
averaged to provide a representative measure of (ability-adjusted) student 
achievein&nt 'in that R unit. 

rffter aggregation, I & R uni^: means on residual Scores from achievement 
Subtests were combined to form an overall reading achievement score and an 
overall math achievement score. Recall that overall scores could not have * 
been formed for individual students because, with a few exceptions, a given 
, individual took only some <jf the subtests. The overall reading and math achieve- 
ment scores--f orme<J by simple addition of I & R unit averages, of ability-adjusted 
subtest scores — are the primary achievement measures* used in later analyses^ 

Tabl^, ^ and 7. p^royide descri^ive statis'tit:s about the univariate dis- 
tributions of all ability and achievement measures, using the individual pupil 
as the unit of analysis. Tables 8 and 9 present -descriptive statistics, 
using the I & unit as the unit of analysis. 
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Table 6 

Grade 2 Student 'Ability and Achievement 
Descriptive ^atistics for Individuals 



T 



Valid Standard 
V^^^^^Q N Mean deviation Minimum Maximum 

Short Form Test of Academic Aptitude (SFTAA) 

Language ability (raw score) 5,346 29.688 6.517 

Language ability (adjusted by 5,349 . 29.724 6.436 - 8.368 45.000 
winsorization) 

Nonlanguage ability (raw 5,312 26.732 6.678 

score) #* 

Nonlanguage ability (adjusted 5,349 26.747 . 6.583 8.674 ' 40.000 

by winsorization) 

\ 

Comprehensive Test of Basic Skills (CTBS) ^ V* 

Reading vocabulary i,391 22.694 7.667 4.000 33.000 

Reading vocabulary (adjusted^ 1,380 - .133 . 6.210 -22.347 . 15.158 

Reading comprehension of 1,391 14.352 6.939- 1.000 , 23.000 

sentences 



Reading cbmprehensiOT of 1,380 - .122 6:^30 " 18.135 

sentences (adjusted) 



12.758 



Reading comprehension of 1,367 11.191 5.539 .000 18 000 

passages , ' . 

Reading comprehension of 1,352 - .034 4.616 " -l"3!946 —11 643 
passages (adjusted) 

Mathematics computation 1,376 13.688 5.643 * 1.000 28.000 

Mathematics computation 1,362 - .044 4.924 -12.156 17 047 
(adjusted) 

Mathematics concepts and 1,402 15.361 5.072 2.000 25 000 

applications 

Mathematics concepts And 1,390 - .025 3.857 -12.432* 13-^.065 
applications (adjusted) , ' ' 
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. • Table 7 
Grade 5 Studej^t' Ability and Achievement 
Descriptive Statistics for Individuals 



Valid Standard > 

Variable Mean deviation ^JCJifiiinum Maximum 



Short 'Form Test of Academic Aptitude (SFTAA) 

Language ability (raw scored ' 5,578 21.209 8.520 1.000 45.000 

Language ability (adjusted by' 5,604 21.201 8.501 1.000 45.000 
winsorizati'on) 

Nonlanguage ability (raw 5,586 24'.264 6.953 3.000 ' 40.000 ^. 
score) 

Nonlanguage ability (adjusted 5,604 24.272 6.911^ , 4.000 40.000 
by winsorization) 

Comprehensive Test of Basic Skills (CTBS) — . , • 

Reading vocabulary 1,477 24.979 8.129* 3.000 40.000 

.Reading vocabulary (adjusted) 1,468 - .118 4.564- -22.974 15.715 

'Reading comprehension . • ". - ^ -1,450 27.196. 91523 2.000 45.000 

Reading comprehension 1,436 - .045 5.622 x -27.797 ^"^5.596' 

(adjusted) . ' I 

Mathematics computation, , 1,416 28.855 9.022 5.000 ' 47,000 

Mathematics computation^^' '"""1,401 .057 .6.832 -21.112 2l'rD73 - 

(adjusted) . ^ " - 

Mathematics concepts ' 1,423 15.106 4.4^1 ^ 2.000 25.000 • ' • 

Mathematics concepts ' 1,398 .,-.026 2.940 -.-11.802 9.250 

(adjusted) » ' 

Mathematics applications- 1,422 14.507 5.818 .000 • 25.000 

Math^atics applidations 1,397 - .-009 . J. 921 -17.053 11.8a7 « 



"■(adjusted) 



Table 8 

Grade 2 Studer^t^||^lity and Achievement 
Descriptive Statistics for School Aggregates 



Valid Standard 
Vari^le N Mean deviation Minimum Maximiim 



Short Form Test of Academic Apti|yde (SFTAA) 

'Language ability (adjusted 156- . 29.794 2.929 19.708 36.944 

by winsorization) 

Nonlanguage ability (adjusted 156 26.814 2.865 16.253 32.000 

by winsorization) 

Coroprehensive Test of Basic Skills (CTBS) 



Reading voccdDulary 


155 


22 


.683 


4 


.071 


8 


.167 


30.500 


Reatding voccQDulary (adjusted) 


155 




.168 


3 


.123 


-9 


.521 


6.521 


, Reading comprehension of 
sentences 


155 


14 


.327 


3 


.132 


.6 


.000 


21.308 


Reading comprehension of 
sentences (ad justed) 


155 




.152 


2 


.669 


-8 


.081 


5^547 


Reading comprehension of 
passages 


^55' 
« 


11 


.107 


2 


.814 


3 

4 


.429 


16.333 



Reading comprehension of 155 - .121 2.065 _ -5.244 5.645 

* f9assaqes 

Q Wa^ematics computation ' 155 13.619 3.036 5.625 23.500 

Mathematics computation 155 - .121 2.545 -7.355 10.286 

(adjusted) 

Mathematics concepts and ' 154 15.375 2.715 8.571 21.100 

applications > 

matics concepts and 154 -.041 .1.870 -6.323 3.647 




plications (adjusted) 

Reading total score • 15=4 48.1L4 8.985 21.000 65.538 

Reading total score (adjusted) 154 - .454 6.687 -18.909 12.402 

Mathematics total score, 153 29.007 4^,869 , 16.054 43.856 

Mathematics total score 153 .260 3.470 -19.857 7.627 
(adjusted) . 

4 ' ' ^ » . 
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Table 9 

Grade 5 Student Ability and Achievenent 
Descriptive Statistics for School Aggregates 



, variable 


Valid 
N 


Mean 


Standard 
deviation 


Minim iM 


Maximum 


\ Short Form Test of Academic Aptitude ■ 


(SFTAA) 








Language ability (adjusted f-- 
by winsorization) 




21.135 


3.317 


9.375 


29.286 


Nonlanguage ability (adjusted 
by winsorization) 


156 


24.213 


2.879 


15.072 


30.833 


Comprehensive Test of Basic Skills (CTBS) 




<• 




Reading yocabulary 


156 


24.902 


4.134 


11.818 


32.000 


Reading vocabulary (adjusted) 


156 


- .137 


1,948 


-9.172 


4.671 


Reading comprehension 


155 


27.044 


- 4.571 


13.000 


37.400 


Reading comprehension 
(adjusted) 


154 


- .117 


2 .465 


-10.685 


6.348 


^ Mathematics computation 


1^ 

153 


28.834 


4.347 


15*833 


38.125 


Mathematics computation 
(adjusted) , 

Mathematics concepts 

>^ 

Mathematics concepts 
(adjusted) 


• 009 


3.305 


-8.993 


11.547 


154 
l53 


15.057 
- .011 


' 2.274 
1.332 


8^.400 
-3.963 


20.500 
3*072 


Mathematics applications 


154 


14.457 


2:. 914 


6.000 


20.833 


Mathematics applications 
(adjusted) 


153 

> 


- .044 


1.706 


-6.075 


4^229 


Reading total score 


155' 


$1,935 


7.655 


24.818 


67.100 


Reading total score (adjusted) 


154 
152 


- .260 


3.470 


-19.857 


7.627 


Mathematics total score 


58.400 


7.808 


34.400 


76^042 


Mathematics total score, 
(adjusted) ^ 

— 9 


152 


- . 097 • 


4.597 


-11.823 


14.019 
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Results and Discussion 

Reading cind Math Achievement 

This section examines features of IGE schools believed to affect achieve- 
ment in reading and math. The positive relationships between features o£ IGE 
schools and measures of students' achievement (as implied in the model) were 
not found. In grade 2 I & R units, no orgamizational variable and no measure 
of instructional practices was correlated beyond a trivial level with either « 
reading achievement or math achievement, although two correlations were statis- 
tically ^ignific2uit. Reading achievement had a correlation^of .197 (p <^^1) 




ERIC 



with General Staff Background (GSB) , cuid math achievement had a correlation 
of •255 (p <^1) with General Implementation of the Instructional Programming 
Model. ^ ^ . 

The results in grade 5 I & R units were just as negative. Every variable 
was correlated only trivially with the reading achievement and matfi achieve- 
ment variables. Not a single correlation differed, from zero with statistical 
significance. Accordingly, the structural equation analysis lendssno empirical 
support to liypothesized paths of influenced 

This negative binding may, in part, indicate faults in our measurement of 

^ 4 

organizational features^^-and instructional practices. However, not all of the 
blame can be placed on the attenuating effect of noise in the measures, because 
ratings of organizational features made by field observers in 30 schools showed 
reasonably high correlation with the questionnaire-base'd scales (lOR, .66; 
GOS, .53; lOS, .60; SIPMi .67). There is less assurance that ou^ measures of 
instructional practices are trustworthy. 

The stcindardized^ group-administered measure of rea<^pig achie^^^^ent cam 
also be criticized as a less-than-j5erfect indicator of what children know 

61 \ 
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about reading. The test may have some j sections that do not reflect the cur- 
riculum of the schools in our study. On the whole , 'however, the instrument 
seems to provide a reasonable assessment. Other information we h^ave indicates 
that the test scores are, reasonably accurate. Different demographic classi- 
ficati^ons of schools, for instance, show differences on the reading achieve-^ 
ment and math achievement measures that closely parallel the findings in the 
National Asse^sm^nt of Educational progress. 

Organizational Features and^Instructional Practices 

The following discussion examines linJcs between organizational features 
and the instructional practices they were meant to facilitate. Analysis began • 
with an estimation of the a priori model.., The initial estimation procedure 
was a straightforward application c^f multiple regression. , The correlation matrix 
for schoolwide measures is given in Table 10. The correlation matrices of addi- 
tional variables used in the regression analyses are ^iven in Tables 11 and 12; 
summary statistics for these variables were given in Tables 3 and 5. Each endo- » 
genous variable was regressed on its theorized causes. Endogenous variables', 
those hypothesized to be under the influence of other variables in the model, have 
at least one straight arrow pointing toward them in a causal diagram. Exogenous 
variables are those not under the influence of other variables in the model. 

Adjacent to each path in Figures 3, 4, 5, and 6 are path coefficients 
(standardized multiple regression coefficients) as estimated for the full a 
priori models of grade 2 reading, grade 2 maj^, grade 5 reading, and grade 5 
ma,th, respectively. These coefficients were subjected to a statistical test. 
The a priori model was revised on the basis of those tests. Causal links 
hypothesized- in the a priori models whose path coefficients did not differ 
statistically from zero were dropped from the models. In effect, such a pro- * * 
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Table 10 

Correlation Matrix for Schoolwide Measures 

e 

(N = 156 schools) 





\ 


SIPM 


lOR 


GOS 


I OS 


GSB 


' INDIV 


SIPM 




• ■ r~ 

1.00000 


f— 

• * 










lOR 




" .60854 


1:00000 








GOS 


1 

I 


.72397 V 


.72345 


1.00000 








lOS 


.49570 


.46323 


.56988 * 


1|. 00000 






GSB 




.25864 


.23079 


.24814 


.27759 


•1.00000 


* > 

1 


imiv 




.45289 


.43155 


.47223 


.20199 


.16266 


1.00000 
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, Table 11 








1 






* 


Reading Correlation Matrices for Reading Analysis. 






»^ 




lOR 


GOS 


lOS 


GSB 


INDIV SIPM 


UCRREAD lAREAD 


IDMREAD 


lEREAD 


ACH • -READ 












, Grade 2 










UGfeREAD 


.15205 


• 17846, 


•12725 


•15381 


.11666 ^21802 


1^00000 




/ 




lAREAD 


•15507 


•25631 ' 


•11606 


•06925 


• 29210 :'31110 


•13904 1^00000 








tOMREAD 


•18010 


•26506 


• 21058 


.05681 


•09314 ^25806 


•24168 ^21317 


i^'ooooo 






lEREAD 


•10159 


•13499 


•05800 


•02489 


•16992 ^21395 


•11377 ^15357 


•35616 


1^00000 


-ft 


ACH.-READ 


-•06695 


-•10528 
* 


-•07779 


•19749 


-•11062 -^63360 


-•08624 ' r^04865 


•00786 


-•01926 


1.00000 

■** "* ^ 

■/ 












Grade 5 ' 


♦* 






) 


UCRREAD 


•17408 


•16749 


•03755 


•15325 


•16385 ^20324 


1^00000 








lAREAD 


•15821 


•14556 


•22917 


•07371 ' 


•25673 ^18552 


' ^10194 1^00000 






» 


, IDMREAD 


•18543 


•13222 


•26817 


•24024 


•25756 ^26017 


•15875 • 33666 


!• 00000 




• 


lEREAD 


, • 14491 


•01421 


•00291 


•02329 


•17535 • ^13441 


•16371 ^21933 


•28312 


1^00000 




ACH.-READ 


•12003 


• 05247^ 


•01872 


•09591 


r 

• 14392 ^(•01230 


•06137 -^05028 


-•03345 


-•07301 


1^00000 




■ *- 


















, Note 


: Correlations presented are 


by pairwise 


deletion • For grade 2, 149 < N < 154; 


for grade 


5\ 147 < 


N < 155^ 

— — r 






* 






r 


< 
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. Table 12 

Correlation Matrices 'for Mathematics Analyses 





TOR 








INDIV 


SIPN 


UCRMATH 


lAMATn 


IDMMATH 


lEMATH 


ACH . -MATH 












Grade 


2 












UCRMATH 


.05461 


.18196 


.04341 


.08216, 


' .16589 


,12369 


1.00000 




* 






lAMATH 


' .18512 


.28016 


.17778 


.10369 , 


.33295 


.30899 ^ 


.^2524 


1.00000 




• 




^DMMATH 


.09937 


.23457 


.22372 
.14559 


.07899 


. 18472 


.22697 


.06024 


.16786 


1.00000 






lEMATH 


.07139 


.14405 • 


.07569 


.17832 


. 18894 


' .01940 


.27100 


.32554 


1.00000 


.00362 


ACH.-MATH 


.08778 


.08633 , 


.06342 


.07185 


.13485 


.25525 


-.05999 


.03532 


.10130 


.00362 


1.00000 










* 


Grade 


5 




• 






• 


UCKMATH 


•10950 


.20235 


.02094 


.06060 , 


.15819 


.15175 


1.00000 


• 








lAMATH 


•16362 


.14873 


.11683 


' .01812 


.22176 


. 086251. 


.15141 


1.00000 








IDMMATH 


. .10836 


.09788 


.18035- 


.20125 


.16347 


.20342» 


• .11955 


.24383 


1.00000 






lEMATH 


.10236 


.02448 


.03116 


-.05289 


.01'^6 


.05719 


.20083. 


.24691 


. 19905 


1.00000 




ACH.-MATH 


.13695 


-.00995 


-.06048 


-.01086 


-.07241 


.03432. 


.03183 


-.02593 


-.06153 


.01654 


1.00000 


Note: 


Correlations presented are 


by pairwise ^deletion. 


For ^rade 2, 149 




for grade 


5, 147 < 


N < 152. 
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Figure 3. Estimated model for grade 2 reading achievement 
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Figuxfi A\ . Bst^nated model £07 grer^e.5 reading schi^vement. 
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cedure tests whether a variable used as a predictor accoilihts for any unique 
variance in the dependent variable. 

Paths theoretically precluded froja an a^^priori model were hypothesized to 

* ' 2 « 

make no increment to R beyond that obtained by using the paths included in 

the 'model. The cmittsed paths of greatest .interest lea^ directly from remote 

^causes to effects — bypassing theorized mediators. When the addition of a 

direct* path from a remote cause — a path theoretically precluded heretofore-:^ 

was statistically signif iccint, ^a model. was deemed insufficient to explain the 

observed relationship between the effect and that remote ca'use. Figures 1 , 

8, 9/ and 10 show^the empirically revised (fitted) models. The information 

in^ these figiires can^'also be represented algebraically as structural equations. 

-Structural equations for ^the hypothesized models and the fitted models ar% 

r ' 

given in Tecfmical Reports 510 amd 511 (Price, Jcinicki, VanDeventer, *& 

Romberg, 1980a, 1980J3) . 

However, when predictors are interrelated — as they -are here-^interpretation 

< . $ 

of the tests for particular path coefficients can easily be misleading (see 
Goldberger, 1964; Gordon, 1968). As Tables if ^nd 12 make evident, the or-' 
ganizational features in this. study were interrelated, as were the instruc- 
tional variables; ^erefore, the substamtive interpretati<in of individual 

path coefficients becomes ambiguous. For this reason we are being deliber- 

^ * * *' * 

ately wary of discussing -individual regression coefficients. 

To get a clearer understanding of the relationships bet:weeirth^ organisa- 
tional and instructional variables, aho^er type of cinalysis was used. ,We 
used orthogonalized predictors , which are^ unrelated to each other ai>d a?re 
linear combinations of the initial set: of., predictors^ Interpretation of . 
coefficients of orthogonalized predictors \s possibleWily to the extent that 
.th^ orthogcmalized predictors can be defined in a substantively meaningful way. . 
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Figure 9.. Fitted model for second-qrade mathematics. 
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■ . • N . ' . * ' : _ - " 

Orthogonal izatkon of organ i^e^tional variables . The si:t organizational- 

variables in the model were transformed through principal .components factor 

. analysis into a set of six uncorrelated (orthogonal) factors.' Varimax rota- 

, tion was used on the full set of principal components. .The netief'fect of 

. . • • " * ' \ • • •' ' • . 

performing these transformations' wks .to- impose orthogonality on the set of / 



six predictors e -while preserving as 61osely as possible a 'on^-to-one corres- 
/ } ♦goii<9ence v^ifiable^ and 'jJifincipal* cdmponenJ:s. J Correlations Jt)etween 

1 \ *^ the .six orthogonal ^factots and the six' .untransf ormgdLvariables. are given in 
^-r^abre l.:^;- , these *cor.relatidns are ^ ispeci^l type^of factor loading. As is , *^ 



evidfenrt: in Table^ 13, each of the si-x factors is a good proxy ^or one> of the 
> . • ' • - ^* 

. variables and -is negligibly related to the. other five variables.. Thus, two 

orthQgohalized*f>redic tors are substantively meaningful, so .their, coefficients- 

^ ' ' ' * ' . * 

in a- regression equation do permit- some interpretation. 

When ^predictors are orthogonal like this, standardized regression coef-, 
ficients are equivale^nt to correlation coefficients* Therefore r results based' 
on orthogonal ized predictors are presented as correlation 9oef ficients* 

OrthogonaMzatioh of ^Instructional variables' . For each of the four 
HK^dels (grade 2 -reading, grade '5 reading, gradg 2 math, grade 5 math), tl^e 

four instructional variables in the model were transformed through principal 

" * * t, 

components factor analysis into a s^of four uncorrelated' (orthogonal) factors. ^ 

/ > ' ' ^ ^ ' : § 

Correlations between the un transformed variables of each model and J:he corres- -'f % 

' • \ ' ( ^ • i 

ponding orthogonal farctors are ^iv^n in Table 14 •* 



Ojr relations between the organizational factors of Table 13 and the 
instructional factors of the respective models are given. in Table 15, As i 
\ • can be s^en from' this' table, the cbrrel'ations between the organizational 

d ■ 

and instructional factors are generally nonsignificant; even those that are 

ERIC, ■ • • ■ 8o' . r ■ 
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Table ' 13- - ^ 

Correlations Between the Organizational ^Variables and' 
^their Vardjnax~r6tatec3^. Principal Components 

' : — ' ' • ^ y ■ 

' ' ^ * " Factors ' ^- / ' 



Variables 


GOSf AC 


. lORFAC 


IPMFAC 


INDIVFAC 


' lOSFAC 


GSBFAC 


-^^^ '■ 














dos 




. .38 


.36 - 


^ .23 


.29 


.10 " 


lOR ; 


.24 


.89 


.25 


.20 


.20 


'.09. 


IPM 


.25 


.26 


.87 " 


* .22 


.23 


.12 


INDIV ' 


.14 


• 1-7 . * 


.17 


.96 


.06 ' 


..06 


lOS 


.06 


■ .18 


.!l9' 


.06' 




.13, ' 


GSB 


.06 . 


•08, ^ 


* .09. 


.06 


• .ii' ' 


.98 


Eigenvalue of 


.72 


1.07 


r.02 


1*.07- 


1.08 


1.01 


factor 
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Table 14 



■j. . \ ■ 

Correlations Between Instructional Feature Variables 



and their Varimax-rotateH Principal Components 



Factors 



Factors i 



ERLC 



Variables * 


lEFAC ^ 


UCRFJ^C 


'lAFAC 


IDMFAC 


^ Variables • 


LEFAC 


• UCRFAC . 


lAFAC 


IDMFAC ' * 




> 

Grade 


2 'Reading 






m--~ — 


Grade 


2 Mathematics 


* 


IE ^ 




.05 


.06 


.17 


IE 


• ^ 


.00 


• 

*.14 


.16 • 


.UCR 


.04 


.99 


.07 


.10 


i • 6CR 


.00 


1«00 


: .06" 


,03 


lA 


.06 


.07 


.99 


"".09 


lA 


• .13 


.06 


\ ^ * 


• .07 


IDM 


.18 


.11 


.10 . 


.97 " 


IDM 


.-16 


... .03 


.07 


" .98 


Eigenvalue 
of factor 


1.00 

i— 


i/oo ' 


1. 00 


.99 


Eigenvalue 
of factor 


1 'nn 

J. . uu 


s 

1.0.0 


-1.00 


1 nn 




Grade 


5 Readin'g 








Grade 


5 ""Mathematics 




UCR ' . • 




.07 


.05 ' 


."08 


UCR 


..99 


".09 


.06 


.06 


IE 


.07 


-^.98 


.09 


.13- 

l 


. IE 


•i08 


' .99 , ' 


.10 


;09 


lA 


.05 


.10 


v98 


.17 


' lA^ 


' .06 


..la 


.99 ' 


• .12 . 


IDM 


.08 . 


.13 


.17 


.97 


IDM 


.06 ' 


.09 




.99 • ' . 


Eigenvalue- • 
of -factor 


.99 . 


.99 

I 


1.00 


.99 


/ 

Eigenvalue 
. . • of factor 


1.00 


. 1.00 ' Q 


1.00 


. i.oo' c:; 

— c ^ — ~, 



/ 



Corrections of Organizational and Instructional ..Factors 





1 


Instructional 


1 < 






— — ^ 

" Ins'trqctional 




Organizational 


• UCRFAC 


lEFAC 


lAFAC 


IDMPAC 


Organizational 


^' UCRFAC 


, ; lEFAC , 


lAFAC 


IDMFAC 




Grade 2 


Reading 


« 






Grade 2 Matheftiatics 




Ti 


lOSFAC ' ' 


.04 


-•02» 


.01 


— — - — ■ ■ 
.14< 


lOSFAC 


- 01 


* ,' ' 


.06 


• .19*^ 


lORFAC, 


. 07 * 


;oo 


.00 


.05 


^ lORFAC . . 


r.04 


-,05 


.03 


-.05 


^INDIVFAC 


.07 


" .13 


* ,23* 


,-.Q8 


INDIVFAC 


* \ .11 


. . . 06 ' • •/ 


24** 


\ 

.06 


SiPMFAC 


'.12 


%14* 


22** 


.15*, 


SIPMFAC 


' 05 

■ V./ 


11 ' 


• .19** 


.13* 


GSBFAC • 


. .'13 • 


.00 . 


.01 
.11 


'-.02 fv, 


■ gbsfAc . - .V- 




01 * 


: .OS 


.■04 


GOSFAC 


.04 


• .00 


• 16* 


'GOSFAC 


•V 




•.10" 

> 


.15* 




Grade 


5 Reading 


' — ■€~ 




■ " 


Orade 5 


^thematics-^v. 






lOSFAC . 


--.04 


-.08 


.20** 


.22** 


1 

' .lOSFAC 


. -.02 


\-.03'^^ 


• 10 


.15* 


lORFAC 


.10 


a? .12 


.04 ' 


.05, 


lORFAC 


' , .03 


.09 


'■.10 


-.01 


INDIVFAC . 


.09 


.11 


.2q** 


.17* 


INDIVFAC ' . ' 




t • t • 

. -.03. ■ ■ 
' -02 • '^^ 


-.18* 


.13 


SIPMFAC 


.13* 


• . ^08 . 


.06 


. .15* 


SIPMFAC / 


v. 09 


1 * 

--04 


.17*:* 


GSBFAC * 


• .10 


-.03 


-.02 . 


'.18* . 


/ GSBFAt 


VVoi 


-v08 , ' 




.17* 


GOSFAC 


.08 


t -.16 

/ 


-.01 


-.11 


/ GOSFAC 


.17* 

^ 


-.06 


..07 .'*> 


\-.07 
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statistically significant ari^ nevertheless weak. Our discussion is limited 
to those correlations that were statistically significant" in botH. grades ^ 
and '5 in eitW reading or^math. Those relationships are listed in Table 
16. - Weak but statistically significant , correlations occurring in 6nly\3ne. 
grade were regarded as undeserving of serious ^attention. \ 
' The cdrrelations in T^le 16 are Weak, but they are statist;Lcally sig- 
nificant. Except for the relationship between GOSFAC and UCRFAC, which was 
statistically significant at. both grades in mathematics only, the other three 
relationships are/ statistically significant in both reading and mathematics 
at both grades 2 and 5". Thus, although' those correlations are weak, they dp- 
persist at a statistically significant level in four out of four circumstances. 

Two of the persistently significant correlations involve- the orthogonalized 
measure (IDMFAC) of the I & R unit ' s. Indiyidualization o^ Instructional Deci- 
sions (im). I s R units Jthat^scored high on IDM tended/to be part of^ a- ' 
school that scored high on sGhpolwj.de .Aspects .of the implementation of the 

% • » \ • 

Instructional Programmi-ng Model (IPM) as indicated by the persistent correla- 
tion between SIPMFAC and IDMFAC. This ' finding i's consiktent with the assump- 
tion made in IGE that schoolwide implementation of the IPM affects its imple- 
mentation at the level. of individual. I &.R units. I &'R'units that scored 
high oi^ IDM also tended -to be part of a school that conformed to the i'ntra- 
organizational structure recommended for IGE schools, as indicated by the 
persistent correlation between lOSFAC and.IDMFAC. 

Th6 other relationship marked by persistently significant ^jo'rrelatiops 

is that between INDiv (a measure of , the extent tb which teachers in a school T 

•» 

believe that individual differences are important to consider when making 



• ' ' Table 16 
Bivariate. Relationsfaip^ that had Statistically 
Significant Correlations ^n Both G3?Ades 









Correlations 






Grade 


2 ^ 


Grade 5 






Rte^ding 




✓ 




lOSFAC & IDMFAC 






,22 



I 



INDIVFAC lAFAC * ^ ^ .23 ^.20 

r I 

SIPMFAC''& IDMfAC ^ .15 % -^^ 



• Mathematics ^ ^ ^. 



. - A- 









lodl^AC & IQMFAC • , 


:i9 


.15 








s 






INDIVFAC & *IAFAC . 

♦ 


.24 


. ■ .18 


SIPMFAC & IDMFAC - 




.17 


ft 






GOSFAC & UCRFAC 


.17 


1 .17" 








. ' ^ * . i 













in^tryctiohal* decisioi>s) and IA^(a me'asure of the extent to v?hx-ch an I & R 

• unit collects informatlion about individual diff fences in reading and mathe- ' 
matics, respectively). The correlations supporting this statement are those 
between tl^e* orthogonalized^ measures INDIVFAC and lAFAC. Although these cor- 
relations aire weak, they^are persistently significant;^ and they do suggest 
th^'t t^.achers who believe in 'the instructional importance of- iildxvidual 
differences are rtio^e likely to collect information- of a kind nihat* will ' ^ 

i support the^ in(3ividualization of instructional decisions.' ' 

As , mentioned^ earlier , the correlation 'between- GOSFAC and'uCRFAC was siqni- 

^ ficant at 'both grades, 2 arnd '5 only in mathematicsr. /The " a priori *mode I did pre- 
diet positive correlatio^Kt^tween G'oS (Procedures Fostexirlg Coordination and" 
Improvement o^the School Program) /and UCR (Utilisation of ipM-gompatllDle 

' .Curriculum Resource^ by I & R Units), but the model predicted the positive « 
correlation in both reading and mathematics, not'jUst in^matheniatics 



Jpb' Satisfaction 



The premise that^teacher job satisfaction- is affected by particular or- 
-ganizatipnal'features' of -IGE has been represented as a network of posliulated 
causal links among- t'fe variables. Figure li presents' the various causa; .finks 
in diagram form, /it showfe' the paHhs of influence assumed a priori tx) underli-e* 
the relatiionships between teacher job satisfaction and the other variables v 

Four variables, Interorganizatiohal Relations (lOR) , Procedures Fostering 
Coordination and Improvement of th^ School*. Program (COS), Intraorganizational 
Structure (lOS), and General Implementation of the -^structional Programming / 
Model (SIPM) , are hypothesized to, relate to teacher job satisfaction. The. 
model in Figure 11 suggests that the amount of communication with other IGE 



79 



schools (Ar) should helj/ teachers feel better about their skills and be mare 
satisfied with teaching. The structural model also suggests that schools with 
a well-f\inctioning IGE program (schools hi'^h on QOS, lOS, and SIPM) would have 
satisfied teachers. The model indicates that General Staff Background (GSB) 
and Belief in Individual Differences (INDIV) indirectly affect teacher job 
satisfaction by way of SIPM. Factors affecting SIPM are discussed more completely 
in Technical Reports 510 and 511 (Price, Janicki, Van Deventer, & Romberg, 1980a, 
1980b) . . • , 

As with the analyses presented already, the analysis began with an esti- 
mation of the a priori model. The correlations of schoolwide variables with 
the job satisfaction variable are given in Table 17. Each endogenous variable 
was regressed on its thfeordzed causes. Figure 12 shows the coefficients as 
estimated for> the a priori model for teacher job satisfaction. ' • 

As a set of predictors, organizational features in the a priori model 
account *fpr 29.1% of the vari^ince in teacher' job satisfaction, which is. sta-' 
tistically -significant (£ < .001). For stpdies of this -kind, 29.1.%,^of ' the 
variance is good prediction. Whether on^ should regard good empirical piredic- 
tion like this as practically significant depends on the interpretation. given 
to the empirical relations, however strong they are.^ By' adding to ^ the predic- 
tion equation exogenous variables that had not been included a piriori as pre- 
dictors of teaeher job satisfaction, the multiple R is .301, which is only 
a 1% increase in variance explaine3. That small increase is not statistically , 
significant (£ < ^25). Since the a priori model is approximately as predictive 
as is the fuller model, we are inclined to judge the a priori model to be 
adequate. , . . ' ? 
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Table 17 

; * Correlations of Schoolwide Variables^with 

the Job Satisfaction Variable 



Schoolwide variables , " JOBSAT. 





SIPM 


.41421 




lOR ^ 


.45554 




GOS . ' 


.49912 




lOS 


.18243 




GSB 


.04655 




INDIV * 


.32409 



106 



"3 
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However, the negative '^coefficient of lOS does give '4ome pause; the "a priori 

model would have tl^at coefficient be positive. The zero-order correlation of 

lOS with JOBSAT is,' in fact, positive, ^ It is .18 (£ < .05, one-tailed test). 

' • * * J * 

The negative coefficient is inconsistent wi^th the interpretation implicit in 

the a priori model. The contradictory interpretation -u.s that the features * 
measured by lOS have a weakly negative effect on JOBSAT, which is not apparent 
by examination of * zero-order correlations because of the Countervailing ' 
influence of lOR, GOS, and SIPM, with which lOS is positively correlated. 
However, neither the a priori interpretation nor the contradictory interpre- 
tation are unequivocally supported by^ these results. ^ ^ • 

As noted in earlier analyses, the* intercorrelatidn of predictors makes 
arly interpretation of ' particular coefficients treacherous.' For that reason, - 
the procedure of orthogonalizing predictors was again used. Variables causally 
prior to JOBSAT in 'the model were transformed through principal components 
factor analysis into a set of uncorrelated (orthogonal) factors. This ^pro- 
cedure, its requirements,* and its limitations were described earlier. 

The uncorrelated, substantively interpretable factors defined in Table 13 
were then used as predictors of '-JOBSAT. By having predictors that are un- 
correlated among themselves, path coefficients can be compared and tested 
without apbiguity — as long as the var^.ables used as j^edictors are themselves 
substantively meaningful. 

Organizational features- as predictors . Factors correspondGig to three 
organizational features had correlations with JOBSAT that were statistically 
significant and of noteworthy magnitude. The factor corresponding to GOS had 
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a correlation of .32 (£ < .0001, . one-tailed, test) ^yi-th JOBSAT. ^The factor 

corresponding to lOR had a correlatioa o£..31 (p < .0001, one-tailed test) ^ 

and the factor corresponding to SIPM had .24 (£ < .01, one-tailed test). The 

zerbTorder correlation coefficients reported here^are equivalent to the 

standardized regression- coefficients of JOBSAT regressed. on the factors^ 

2 

corresponding to GOS, lOR, and SIPM (R = .261). That equivalence is inherent 
to perfectly uncorrelated predictors. The zero-order correlation of the factor 
corresponding to lOS, another organizational variable, is .b2, wjiich is statis- 
•tically nonsignificant and trivially small. 

School, staff measures as predictors . Besides the four organizational 
features discussed in the preceding paragraph, two measures of the school- 
staff, INDIV'and GSB, were treated as causally prior to JOBSAT. The correla- 

A 

tion of the factor corresponding to INDIV with JOBSAT is .20, which is statis- 
tically significant (£ < .01, one-tailed test). The correlation of the 
.factor corresponding to GSB with JOBSAT, on the other hand, is -.03, which 
is trivially small and statistically nonsignif icant. The addition of the 

factor corresponding to INDIV to the pr^ictor set of GOS, lOR, and SIPM im-. 

' ^ 2 2 

proved the prediction of JOBSAT from R = .261 to R = .299, a statistically 

significant increase. 



Conclusion 

The concljision of this report considers the findings of the Phase I study, 
dividing them for sake of exposition inta five' parts:' 

!• Relationships between organizational features and instructional' prac- 
tices qf IGE schools. A . • • « 

2. Relationships between staff beliefs and instructional practices-.' 

3. Relationships between instructidnal practices of IGE schools and 
measures of students' achievement. 

4. Relationships between organizational features of ^GE schools and 
students' achievement. ' ^ 

5. Correlates of teachers' job sati^sfaction in IGE 'schools. i * 

Organizational Features and Instructional Practices 

Part of IGE Schooling consists Of organizational featur^es designed to 
facilitate the instructional practices that compose the Instructional Pi^o- 
gramming Model. , Because those organizational features have as a-primary 

purpose the facilitation of certain iiistructional practices, the •^Phaseu I 

• . • • /> • 

.study examined empirically the^ implied relatd^ons between, organisational 

feature? and instructional practices Some implied correlations between 

organizational features and instructional practices were borne out in the ^ 

Phase I data;, others were. not. The implied relations that were bori>e^out 

• * ' * ■ 

empirically were presented in Table 15. The practice of individuali?;ing 

< *' 

instructional decisions — an instructional practice pursued to varying de- 

* ' • * 

grees by I & R units- in the scJiools studied — does seem to be facilitatfed • 

' / - • 
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by certain schoolwide organizational^ features, in particular t^he schoolwide 
implementation of the Instructional Programming. Model and- the intraorganiza- 
tional structure of the schools. " * - • 

Schoolwide implementation of the IPM . The extent th which the Instruc- 
tional Prqgraijjming Model (IPM) ^had been implemented by the school In general 
(and not simply in the I & R units studied) was positively -correlated with 
the degree to which the specific I & R units under, study engaged in the in- 
divick^alization of instructional decisions. This finding may surprise no 
one, but it does* attest to- the susceptibil^ity pf small groups of teachers 
to the larger (schoolwide) milieu in which they are situated. i 

Intraorganizational structure . Another aspect of the larger milieu* , 
that was consistently associated with I & R units' individualization *of 
instructional decisions is the intraorganizational structure of the school;- 
(This .is a proxy for various structural arrangements distinctive to IGE * 
schools, such as the organization of staff and students into I & R units ' 
and the existence of a functioning Instructional Improvement Committee.) 
These correlations with I & R units' individualization of instructional de-' . 
cisions offer' some vindication of the organizational theory contained in the, 
IGE system. According to thdt theory, indivi'i^uals who are part of an ..organi- 
zation are affected by controllable features of that organization. 

Implied correlations not found . , Some /implied correlations between 

• ~~' ' ' y jl' ' . ^ ; 

organizational features and instructional practices were not borne out by 

the data in a. consistent fashion. They were: / . ^ 

1. an expected connection between the interorganj.zational relations 

of a school and the uti'lizatioji of IPM-compatible curriculum materials by 

I & R xinits in that school. .1 ^.V) 

* * * •> * , *^ *w ' 



2, expected connections .between schoolwide implementation of the IPM* 
and . , ' ^ ^ V , J 

a. ^utilization of IPM-'compatible mdtei^ials by I & R units, 



b* collection of . information about individual differences, '*/ 
'c. the I & R unit's management of grouping and instructional' con- 
tinuity* 

; ^ ^ * • ' ' % , 

In pa^es such as these, wher^e expected relations were not found, t^ree 
types of explanations 'can be offered. The first might be called a "mo^tel- 
blaming" explanation, .because it faults t?he underlying mocJ^l that has f Sailed 
under test. The second and third might be called "test-blaming," because ^ - 
they fault the procedures that have been used to perform the tfest of, the 
model. The s^ond attributes the lack df observed relation to faulty measute- 
ment of the predictor varia^bles (in this case, m^feasures of schoolwide organiza- 
tional features) . This explanation seems to iTfave little ground in this pase 
for a couple of. reasons. As 'noted earlier ^ in, IShis' report, the Pha^ I ^. ^ 
njeasures of schdolwide organizational featur.es agreed ^easohably well with- 
counterpart measures obtaine.d ih the Phase Il^Ji^eid validation 'study. , ^ 
Furthermore, other parts of ^ the model were bbrn^ out. by correl&tions involving, 
these variables, a circumstance which should not have arisen if these variedDles 

:\ ^' -r : > \ ' > • ' - r . 

■ • "* ' 

^werfe badly measured?,. . „ * ^ . " ^, 

The '^third type of explanation attributes the lack of pbserved relation . 
to faulty measurement of the predicted variables (in- this case, measures of 
I. & R uViits*' ins^tructional practices) . This type of "test-bl£uning^^ explana- 
tion cannot, seriously di^pute^; - There was no field validation. of ^he^se 

' \'* • ' * 
measures, there was ^abundant opportunity *;E0Ef^dis'Jforted ijj/ormation to^'fepter 




the questioni)air^s,, ahd^.lfhere^^are no redeemingly high correlations^ to suggest 
that these variables werd jnaea&uared' reliably. The measures of I & R units' 
instructional practice s ^reC'^e, jLeast' tr'ustvior thy part of the Phase I study. 
For this reason, we are (id/sin'olii^^ Jto engage, in "mo^el -blaming" when data 
* involving these particular 'jn^s^^es ar« involved. - « 

staff Beliefs and Instruc^Jiional Practices 

The beliefs that stajf members -hold* about the value of IGE are obviously 

important. The instructional practice of c6llecting information about in- 

dividual differences between students in ^onteht: areas (reading and mathe- ^ 
* . ^ . H ' • . ^ . ^ ' 

matics) was, as expected, correlated with a measure of the extejit to which 
teachers in a school believe that individual differences are important to 
consider when making inst;ructional decisions. In the presence of^ other 
factors likely to affect this important .element of .IGE practice, it i^s^ note- 
wbrthy that its*' strongest predictor in Phase I was the extent to which teacl^ers 
in a school believed in wha-t" one could reasonably argue is the most basic 
tenet of the Instructional Programming Model; namely, that individual dif-* 
ferences'are pertinent' to ^instructional decisions. \ 

Instructional Practices aM|fl^ Student Achievement . 

» ~ 0^ ^ * 

. \ ^ ^ . . " 

In. no 'instance — no^ in Reading, not in Math,hTiot in'Grade not in 

Grade 5 — was '^hei^ a statistically significant correlation between a. measure 

•of Instructional practices and a measure of student achievement. Besides the 

. ^ ^ J 

"model-blaifiing" explanation for this, which would fault the Instructional 

Programming Model, there are twb other types of explanation. One attributes 
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• • . ^ 

the weak, relations to bad measurement of student achievement; . the other, to 
bad measurement of instructional practices. Student achievem^t, we believe, 
was -measured reliably* Despite the reliability with which student achieve- ^ 
ment was measured, any standardized, group-administered test can be criti- 
cized as, an ^imp^f ect reflection of what children know^about the, area 
^issessed by the test* Persons who wish to make that criticism of the student 
achievement measures--a criticism of construct validity — ^must .concomitantly 
dismiss any favorable, findingi^ based on outcome measures such^as the^* We 
hav6 already mentioned the low trust we place in our measures of instructional 
practiced, ^^fonsequently, unreliability in those measures may have attenuated 
correlations between instructional practices and student achievement,- For 
that reason, we are disinclined to use this particular negative finding as 
a basis for "model-blanting. " 

. Organizational Features °cmd Student Achievement 

9 

Expected correlations between organizational features and student 
achievemelit were not foiand, despite reliable measurement of both classes^ 
of variables* All measures of organizational features were trivially and 

nonsignificantiy associated with student achievement measures. With regard 

* ', * 

to this negative finding/ a "model-blamtng" explanation is the most plausible. 

1 * ^ 

Sp6ci^j^a\lly, these findings ^indicate that impl^entation-of .the surface 

organizational features with which IGE is commonly- identified offer no 

^assurance at all the student achievement will be raised. Evidently,, the 

instructional effectiveness of I & R units included in Phase I did not 

depend on the degree to which' IGE organizational features had been implemented. 
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in the school of which they were part. This finding (based on quest;ionnaire 
dat-a) resonates with the findings of 'other phases of the IGE Evaluation, all 
of which have converged on the conclusion that surface orthodox'^^ reveals 
little if anything about the value of an educational program. Nor does it 
reveal whether the Instructional Programming Model is practiced in a form 
that would be recognizable by, its 'developers. 



t^ jg^r^ates of Teachers' Job Satisfaction 

V ' " ■ ' ' ■ / 

The foregoing discussion followed a long-standing tradition in educational 
evaluation by gauging the worth of an educational option orf the basis' of its 
effects on the stud^ts' achievement. There do, exist other grounds on which to 
evaluate educational programs. In an era when "teacher burnout*' has become a 
household word, one evident alternative is to evaluate the effects of programs on 
staff morale. On those grounds, three organizational^ features commonly associated 
with IGE fare well. /.Three schoolwide organizational features have positive 
correlations (end positive path coefficients) with the schoolwide measure of ^ 
teacher job satisfaction. Those three . features are: (1) the interorganiza- 
tional relations of the school, (2) the existence of procedures fostering 
coordination and improvement of the school program, and (3) general, school- 
wide implementation of the Instructional Programming Model. Underlying 
these positive relations appear to be two factors. One is that teachers 
like the contact with other adults provided by IGE, especially the contact 
with a professional network that extends beyond their particular school. 
The other is that job satisfaction is derived from a belief that their. in- ' 

structional efforts are effective, a belief canmonly held by- teachers of 

• * 

schools i/i which the IPM had been implemented on a schoolwide basis. 
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